Evaluation of an Electronic Case-based Disease Surveillance System in Karene District, Sierra Leone.

Eldred Olufemi Moore^1,2, Umaru Sesay^1,2, Adel Hussein Elduma^2,3,&, Solomon Aiah Sogbeh^1,2, Anna Jammeh^2,3, Amara Alhaji Sheriff^1,2, Bridget Magoba³, Mohamed Alex Vandi¹, Gebrekrstos Negash Gebru^2,3

 

¹Ministry of Health, Freetown, Sierra Leone, ²Sierra Leone Field Epidemiology Training Program, Freetown, Sierra Leone, ³African Field Epidemiology Network, Freetown, Sierra Leone

 

 

^&Corresponding author
Adel Hussein Elduma, Sierra Leone Field Epidemiology Training Program (SLFETP), African Field Epidemiology Network (AFENET), National Public Health Agency (NPHA).

 

 

 

 

Public health surveillance, an essential function of public health, is defined as an ongoing systematic collection, analysis, interpretation, and dissemination of health data for planning, implementation, and evaluation of public health action [1]. In 2003, Sierra Leone adopted the Integrated Disease Surveillance and Response (IDSR) system [2] aimed to enhance efficient resource utilization for disease detection and reporting [3]. Initially, the operation of the IDSR system relied on paper-based data collection, which limited the usefulness of the system to promptly detect outbreaks [4] including the Ebola outbreak between 2014 and 2015 [5, 2]. The Ebola outbreak further limited the operation of the public health surveillance system [2], highlighting the need to strengthen the IDSR system in Sierra Leone [6].

 

In 2016, Sierra Leone integrated the electronic IDSR on the DHIS2 platform to promote data utilization [7]. In 2018, the Electronic Case-based Disease Surveillance and Response (eCBDS) system was launched [2,8] to enhance timely data collection, analysis, and dissemination for outbreak detection [9]. Currently, every health facility in Sierra Leone is to report through the eCBDS system any of the following diseases, conditions, or events including Acute Flaccid Paralysis (Polio), Acute Jaundice Syndrome, Acute Respiratory Illness (Influenza new subtype, COVID-19, SARs, SARI), Neonatal Tetanus, Anthrax, Poliomyelitis (AFP), Yellow Fever, Cholera, Maternal Deaths, Measles etc. These priority diseases are reported immediately (within 24 hours) they are detected at health facilities. Reporting starts at community level, and then to the districts and to the national level. The eCBDS data is used for high-level strategic decision-making to address public health threats [9].

 

Since its inception, the eCBDS system has achieved notable successes in identifying outbreaks at the national and district levels, including the COVID-19 pandemic [10]. Despite the significant impact of the eCBDS on the healthcare system, there is limited information on the performance of the eCBDS system. Until now, no study has been conducted to determine the system's performance and assess its functionality in Sierra Leone. Understanding the eCBDS's performance and its attributes' effectiveness is critical for designing targeted interventions, strategies, and policies for improving the system's performance. This study evaluates the performance of the eCBDS in Karene district, focusing on its operations and surveillance system attributes to determine if it meets its intended objectives.

 

 

Study Design

 

We used a cross-sectional, mixed-method study design to assess the eCBDS qualitative and quantitative attributes, from October to December 2022.

 

Study population and setting

 

This study was conducted among healthcare workers in Karene district. Karene district, located in the North-West region of Sierra Leone, shares an international boundary with Guinea in the north and locally with Porto Loko, Koinadugu, Kambia, and Bombali districts respectively. The district had an estimated population of 290,313 in 2022, with 13 chiefdoms. The district has 62 health facilities consisting of one referral hospital, 15 Community Health Centers (CHC), 31 Community Health Posts (CHP), and 15 Maternal and Child Health Posts (MCHP). The district was the last to implement eCBDS in Sierra Leone, in 2020. Despite this, the eCBDS system has been rolled out in all 62 health facilities in the district. At the health facility level, the eCBDS is utilized by the IDSR/Surveillance focal person, and at the district level, it is utilized by District Surveillance Officers and Surveillance Data Officers.

 

Selection of study participants

 

We purposively selected and interviewed 15 healthcare workers: eight from community health post, one from the district hospital, four from community health centers, and two from the district health management team to assess the qualitative attributes. We randomly selected 13 health facilities: 12 peripheral health units (four Community Health Centers and eight Community Health Posts) and purposively selected the District Health Management Team and the only referral hospital. Staff with more than one year of working experience with the eCBDS system, residing in the study area, and consented to participate in the interview were selected. The 15 healthcare workers were chosen because their experience was deemed sufficient to provide the information required to answer the research questions. The 12 health facilities were chosen due to limited resources to collect data.

 

Data Collection

 

We used the third edition of the Integrated Disease Surveillance and Response Technical Guidelines and the U.S CDC updated guidelines for evaluating public health surveillance systems to evaluate the eCBDS system [11]. For the qualitative attributes, we interviewed healthcare workers and employed personal observation on the case-based surveillance posters, standard case definition posters, and data analysis charts. During the interview, we used a face-to-face interview technique to administer the pretested semi-structured questionnaires, which were categorized by attributes. For the quantitative attributes, we reviewed health facility registers, all case-based forms and IDSR weekly reporting forms, and the eCBDS application in the health facility's mobile Android tablet. Case-based data in eCBDS system for a period of one year ranging from January to December 2022 was extracted for analysis. We compared data reported at various levels and determined the reporting rates. We assessed usefulness by assessing whether an action was taken on the data generated from the system or whether the data was used to detect an outbreak. Additionally, record reviews of data stored in the Ministry of Health and partners´ website, as well as personal observations, were done to assess this attribute.

 

We assessed simplicity by determining the ease of operation of the system and the time taken to enroll a case into the system. We assessed flexibility by determining whether the system would easily adapt to changes in requirements or user needs: case detection, addition of new data tools, and reporting mechanism without affecting the system operation. Acceptability was assessed by determining the willingness of staff to participate in the system operation, and stability was assessed by determining whether the system experienced any breakdown in its operation. Sensitivity of the system was assessed by considering two dimensions as recommended in the CDC guideline. First, the ability of the system to detect outbreaks, and second, the level of cases reporting, ie, the proportion of cases detected by the system over those recorded in the register.

 

The predictive value positive was determined by assessing the proportion of positive cases to those suspected. Timeliness was assessed by determining the difference in the time; from the time the case reports at the health facility (date seen at the health facility) to the date of registration in the system (date district is notified), if it is <24 hours, and data quality was assessed based on the completeness of filled compulsory variables in the case-based reporting forms in the health facilities. Representativeness was assessed by determining whether the system captures cases in terms person, place, and time. Variables assessed include date of birth, sex, date of onset, date seen at health facility, date health facility notified district, location, and disease conditions/events.

 

Data Analysis

 

For quantifiable data, we calculated the proportion of each question per attribute and then computed the average scores. We adapted an evaluation criteria score used in a study conducted in Sierra Leone and Yemen to assess system attributes [12,13]. We used a Likert scale to rank all 10 attributes as listed in the CDC updated guideline [11]: poor (<30%), average (30 - 70%), and good (>70%). For qualitative data, we summarized participants' responses and reported accordingly. We use Epi Info 7 software to analyse quantitative data [14].

 

Ethics Statement

 

Ethical approval from the Sierra Leone Ethics and Scientific Review Committee was not required, as the study is part of a routine surveillance system evaluation and reporting was done anonymously. However, permission to conduct this study was obtained from the District Medical Officer. Written informed consent was gained from the healthcare workers who agreed to participate in the study. No personal identifiable information was disclosed publicly to maintain participants' confidentiality.

 

 

Socio-demographic Characteristics

 

Of the 15 participants, the majority, 66.7% (10) were females. The median age of participants was 35 years (range 28 - 60 years). The average years of experience among participants were 9 years (range of 28 - 60 years). Among the respondents interviewed, majority, 53.3% (8/15) were state-enrolled community health nurses, followed by maternal and child health aid nurses, District Health Management Team (DHMT) staff, and community health officers with 13.3% (2/15) each, while the least were midwife nurses with 6.7% (1/15) (Table 1).

 

Description of the eCBDS system in Karene District

 

The eCBDS system operates as a passive and an active surveillance system. In the active surveillance the public health workers contact the healthcare facilities to inquire about recent cases of a disease. In passive surveillance healthcare facilities input patient data into the electronic system. The system is developed on the DHIS2 platform for reporting 21 priority diseases including epidemic-prone diseases, vaccine-preventable diseases, prioritized zoonotic diseases, and diseases identified for global eradication and elimination immediately (within 24 hours). The government of Sierra Leone identified communicable diseases, conditions, or events as priorities for IDSR in the country. These priority diseases were selected for immediate reporting during the national adaptation of the 3rd edition of the IDSR technical guidelines [15]. The criteria for the selection of the priority disease to be reported were: Required internationally under IHR such as Smallpox, Polio, COVID-19; Diseases with a high epidemic potential to cause serious public health impact such as VHF, Yellow fever; Vaccine-preventable diseases; Prioritized zoonotic and diseases identified for global eradication.

 

The eCBDS collects demographic, clinical, laboratory, and epidemiological data (Figure 1). The Ministry of Health and partners provided mobile android tablets to all 62 health facilities in the district for reporting. At the community level, the Community Health Worker (CHW) identifies a case and reports to the peer supervisor or health facility staff in the nearest health facility through phone calls or visits to the health facility. The health facility staff (community health officers, community health assistants, state enrolled community health nurse, and maternal and child health aid nurse) classifies the case using the standard case definition, fills the case-based notification form, and then enters the case in the system using an android mobile tablet (Figure 1). A text message or email notification is sent to the district surveillance officer (DSO) or data officer for immediate response. The DSO or the data officer verifies the suspected case with the health facility, makes further investigations, fills the case investigation form based on the disease condition reported, and then enters the data in the system. The laboratory technician collects samples, while the DSO fills the laboratory request form and enters the data in the system. The case investigation can be done concurrently with the laboratory request, the sample is sent to the regional/national laboratory for analysis and confirmation. A text message or email notification is sent to laboratory staff on a laboratory test request. The reference laboratory staff receives the sample and fills the laboratory processing stage in the system. After sample analysis, the laboratory results are entered into the system. The DHMT staff involved in the case receives a text message or email notification on the laboratory results i.e. “The laboratory results for case (XXXX) are ready”. DHMT staff view the laboratory results in the system and communicate to the health facility staff or clinicians managing the case through in-person, phone calls or text message. The DHMT enters the outcome to close the case. Staff at the national level also receive notifications across all stages for their awareness and follow-up. The National Surveillance Officers at the Directorate of Health Security and Emergencies and data managers follow up with the district on the investigation if the need arises, data quality assurance, and analysis using eCBDS [16] (Figure 2).

 

Assessment of eCBDS System Attributes

 

The simplicity of the eCBDS system was rated as good. Of the total respondents (15), 73% (11) said that the eCBDS system is simple to use and few respondents, 13.3% (2/15) indicated that there is an established eCBDS manual or guidelines for data recording, analysis, and reporting at health facilities. Forty percent (6/15) attested they understood the case definition for key notifiable diseases and reported that enrolling a case in the eCBDS platform takes them between 10 and 20 minutes. All respondents (100%) claimed to understand the eCBDS system's reporting mechanisms.

 

Despite its good simplicity rating, the stability of the eCBDS system was rated as average. Disease notification forms were available at all 13 health facilities, and 77% (10/13) of them had a working laptop or Android tablet for reporting. However, 54% (7/13) lacked access to mobile networks and electricity or solar power to charge their reporting devices. Personal observation of the investigators revealed no allocated budget for the operation of the system by the Ministry of Health, and the system did not experience any breakdown in its operation.

 

Regarding flexibility, the Ministry of Health integrated the COVID-19 form in 2020 into the eCBDS system, and this integration did not affect the system´s operation. During our survey in 2022, 40% (6/15) of the respondents confirmed that the eCBDS system can be integrated with other systems. Therefore, the flexibility of the system was rated as average.

 

Although some health facilities had faulty tablets, 67% (10/15) of the respondents indicated that they have a functional tablet/Laptop at the health facility/DHMT facility and have hard copies of the notification form. The acceptability of the eCBDS system was rated as good, with 93% (14/15) of the respondents completing the tasks that were assigned to them. The representativeness was rated as good, as cases were representative of the general population in the district, and all (13) selected reporting sites, including government and faith-based health facilities, reported a case in the system.

 

The eCBDS system's data quality and timeliness were rated as good. Eighty-seven percent (11/13) of the health facilities visited had priority diseases reported in both the eIDSR and eCBDS and only few 43% (3/7) of the variables in the notification forms reviewed had missing entries of the case-based disease reported in the last year (Table 2). For instance, from 32 forms reviewed, date of birth had two missing entries (Table 2). And 100% (13/13) of the health facilities that reported a case informed the district within 24 hours of case detection. The sensitivity of the eCBDS system was rated good, with 100% (32/32) of cases reported in the case notification forms also captured in the eCBDS system. However, the predictive value positive was rated as poor, with only 18.8% (6/32) of the cases submitted to the system being true positive cases.

 

Furthermore, although the system´s simplicity, acceptability, representativeness, data quality, timeliness, and sensitivity were rated good– however, the usefulness of the eCBDS system was rated as average. We observed the eCBDS system was used to detect outbreaks such as anthrax and provided real-time information to guide the response (Figure 3). A review of records in the eCBDS indicated that half of the total cases reported between June and August 2022 were confirmed for anthrax. Also, 93% (14/15) of the respondents confirmed that they were trained on eCBDS. However, only staff from the district health management team, 13% (2/15), were knowledgeable in generating data from the system to perform data analysis. Amongst the total staff interviewed at the health facility (12), only 25% (3) confirmed that they received feedback from the DHMT after sending notifications.

 

 

This study evaluated the performance of the eCBDS system in Karene District to determine if the system is meeting its objectives. This study found six of the ten system attributes assessed including sensitivity, simplicity, representativeness, timeliness, data quality, and acceptability were good. The system's flexibility, and stability were average, while the predictive value positive was poor. Overall, the usefulness of the eCBDS system was rated as average, as the system met its objective of heightening the ability of healthcare workers to detect and respond to notifiable diseases in real-time.

 

Although the eCBDS surveillance system was simple to operate, the lack of an allocated budget for resources, including internet data bundles for reporting, and limited feedback from supervisors to health facility staff, could have demotivated staff for real-time reporting. This finding is similar to a study conducted in Peru where the limited budget allocation for the implementation of an electronic surveillance system, inadequate personnel, and difficulty in system operation were identified as the main challenges of the eCBDS system [17]. A study conducted in Ghana also reported similar findings, where all healthcare workers at the peripheral level do not receive feedback on the IDSR system [18].

 

Although the system stability was rated average, the system's reliance on mobile network access and data bundle for its operation could have affected real-time reporting. Sierra Leone´s electronic surveillance system like many other low-income African countries, depends on funders which makes it difficult to sustain the system. The flexibility of the system, rated as average, was one of the key strengths of the eCBDS system as it allowed the incorporation of COVID-19 cases for immediate reporting. Similar findings were reported in Somalia [19] and Syria [20].

 

Timely detection of disease is a critical component for reducing disease morbidity and mortality as well as disabilities associated with outbreaks [21]. Although the eCBDS system timeliness was rated as good, the lack of functional tablets in a few health facilities affected data reporting possibly delaying the reporting of cases through phone calls and text messages to the next level. This finding is different from a study conducted in Madagascar, where timeliness due to major technical challenges in the surveillance system to respond to surveillance objectives including issues with reporting tools and a heavy workload [22].

 

This study's findings showed that the eCBDS data quality was good. Complete electronic data reporting increases the speed of the response and ensures the reliability of the data collected at the reporting sites [21]. Limited refresher training and guidelines for data recording, analysis, and reporting at health facilities are factors that undermine the quality of surveillance data [23].

 

Sensitivity of the eCBDS was rated good. This could be because the system captures conditions and events that do not require sample collection for laboratory confirmation. Some examples of these disease conditions or events include animal bites (snakes and dogs), maternal deaths, and neonatal deaths, among others. The predictive value positive was poor (18.8%). Our finding was similar to a study conducted in Somalia where the predictive value positive was 14.1% [20] (Figure 3).

 

Strengths and Limitations of the Study

 

Assessing all 10 surveillance system attributes of the eCBDS according to the updated CDC guidelines for evaluating public health surveillance systems is the strength of this study. Despite the strength of this study, the study has two main limitations. First, the purposive sampling technique employed may have introduced selection bias. Secondly, this study was conducted among twelve peripheral health units and one hospital constituting a few healthcare workers. As a result, the findings might not be generalizable to the entire district, and by extension Sierra Leone. However, the findings will be crucial for informing stakeholders in Sierra Leone and other countries with similar settings on developing strategies and interventions for improving electronic surveillance systems.

 

 

The eCBDS system in Karene district was useful in meeting its objective. The eCBDS system demonstrates effectiveness in several key areas, such as sensitivity, acceptability, simplicity, representativeness, timeliness and data quality. However, improvements are needed in flexibility, stability, and particularly predictive value positive. Overall, the implementation of the eCBDS system has significantly improved disease detection, recording, and reporting.

 

To maintain a well-functioning system, we recommend the Ministry of Health, especially the Karene District Health Management Team (DHMT) to replace the dysfunctional mobile Android tablets for reporting and organize regular refresher data entry training and analysis for all health facility staff. The Ministry of Health should liaise with development partners to assist them in the procurement of mobile Android tablets, and the distribution of these tablets should be overseen by the supervisors at the DHMT to ensure they are delivered to needed health facilities. The refresher training should be organized at least once a year. The DHMT should include data entry and analysis training in their annual work plan to ensure a step forward towards securing funding with minimal support from partners. Additionally, Karene DHMT to ensure supervisors provide regular feedback to health facility staff and increase support for supervision and mentoring visits to health facilities to resolve the electronic system technical challenges. Supportive supervision and mentoring should be done at least every quarter. The DHMT should ensure the supportive supervision funds provided by the government of Sierra Leone and its partners are utilize for it intended purpose. During each supervisory visit, the DHMT supervisors should require all supervisors to provide feedback on the performance of the system to the health facility staff.

 

 

The authors declare no competing interests.

 

 

EM and US: conceptualization of the study, data analysis, and preparation of the manuscript. EM: data collection for the study. US, AS, SS, AE, BM, and GN reviewed and supervised the paper. All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved the submitted version.

 

 

We would like to acknowledge the US CDC, AFENET, and Sierra Leone Field Epidemiology Training Programme for supporting this work. I also appreciate stakeholders' willingness and cooperation in giving us the data on the surveillance system. I gratefully acknowledge all who were involved in this study in one way or another.

 

 

Table 1: Demographic characteristics of the respondents, Karene District, 2022

Table 2: Data completeness of case-based data reported in eCBDS in Karene district, January 2022 - December 2022

Figure 1: Paper-based and electronic case notification form used in the eCBDS system, Karene District, 2022

Figure 2: The eCBDS data flow and feedback mechanism, 2022

Figure 3: Number of suspected and confirmed Anthrax cases in Karene district by quarter 2022; Source Sierra Leone Electronic Case-based Disease Surveillance System (eCBDS)

 

 

1. Choi BCK. The past, present, and future of public health surveillance . Scientifica [Internet]. 2012 Oct 23 [cited 2024 Nov 21];2012(1):1-26. https://doi.org/10.6064/2012/875253 Erratum in:Corrigendum to “The Past, Present, and Future of Public Health Surveillance”. Scientifica [Internet]. 2018 Jul 2 [cited 2024 Dec 3];2018(1):1-26. https://doi.org/10.1155/2018/6943062 PubMed | Google Scholar


2. Njuguna C, Jambai A, Chimbaru A, Nordstrom A, Conteh R, Latt A, O-tipo S, Musoke R, Githuku J, Yoti Z, Yahaya A, Talisuna A, Rajatonirina S, Fall IS.Revitalization of integrated disease surveillance and response in Sierra Leone post Ebola virus disease outbreak . BMC Public Health [Internet]. 2019 Apr 2 [cited 2024 Nov 21];19(1):364. https://doi.org/10.1186/s12889-019-6636-1 PubMed | Google Scholar


3. Perry HN, McDonnell SM, Alemu W, Nsubuga P, Chungong S, Otten MW, Lusamba-dikassa PS, Thacker SB.Planning an integrated disease surveillance and response system: a matrix of skills and activities . BMC Med [Internet]. 2007 Aug 15 [cited 2024 Nov 21];5(1):24. https://doi.org/10.1186/1741-7015-5-24 PubMed | Google Scholar


4. Gleason BL, Kamara A, Clemens N, Kargbo D.Establishment of an electronic integrated disease surveillance and response system in Sierra Leone . International Journal of Infectious Diseases [Internet]. 2019 Feb [cited 2024 Nov 21];79(Supp 1):111. https://doi.org/10.1016/j.ijid.2018.11.275 Subscription or purchase required to view full text. Google Scholar


5. WHO (SL). Sierra Leone leads the way in Africa with fully functional electronic disease surveillance system [Internet]. Freetown (SL): WHO (SL); 2019 Jun 6 [cited 2024 Nov 21].


6. WHO. Ebola outbreak 2014-2016 - West Africa: Overview [Internet]. Geneva (Switzerland): WHO; c2025[cited 2024 Nov 21].


7. CDC. What is Case Surveillance [Internet]? Atlanta (GA): CDC; 2024 Nov 20 [cited 2024 Nov 21].


8. Behumbiize P. Evaluation of the implementation and utilisation of a case based diseases surveillance (Electronic integrated disease surveillance and reporting (eidsr-cbs)) in the west nile region: august 2018 to february 2019 . TIJPH [Internet]. 2019 Dec 19 [cited 2024 Nov 21];7(4):1- 15. https://doi.org/10.21522/TIJPH.2013.07.04.Art004


9. Alemu T, Gutema H, Legesse S, Nigussie T, Yenew Y, Gashe K. Evaluation of public health surveillance system performance in Dangila district, Northwest Ethiopia: a concurrent embedded mixed quantitative/qualitative facility-based cross-sectional study . BMC Public Health [Internet]. 2019 Oct 22 [cited 2024 Nov 21];19(1):1343. https://doi.org/10.1186/s12889-019-7724-y PubMed | Google Scholar


10. CDC Division of Global Health Protection. Collaborating to Strengthen Surveillance Systems Worldwide [Internet]. Atlanta (GA): CDC; 2023 Feb 28 [cited 2024 Nov 11].


11. CDC. Updated Guidelines for Evaluating Public Health Surveillance Systems: Recommendations from the Guidelines Working Group. MMWR [Internet]. 2001 Jul 27 [cited 2024 Nov 11]; 50 (RR13):1-35. Google Scholar


12. Al Kalali FSA, Mahyoub E, Al-Hammadi A, Anam L, Khader Y. Evaluation of the national tuberculosis surveillance system in Sana´a, Yemen, 2018: observational study . JMIR Public Health Surveill [Internet]. 2021 Nov 30 [cited 2024 Nov 21];7(11):e27626. https://doi.org/10.2196/27626 PubMed | Google Scholar


13. Sesay U, Gebru GN.Evaluation of a hypertension surveillance system, kenema government hospital, sierra leone, 2021 . Prev Chronic Dis [Internet]. 2023 Mar 23 [cited 2024 Nov 21];20:220230. http://dx.doi.org/10.5888/pcd20.220230 Download pdf to view full text. PubMed | Google Scholar


14. CDC. Epi Info™ [Software]. CDC. [last reviewed 2021 Nov 23: cited 2024 Nov 21].


15. MoH (SL). Technical Guidelines for Integrated Disease Surveillance and Response in Sierra Leone [Internet]. Freetown (SL): MoH (SL); 2008 Jun [cited 2024 Nov 11]. 280 p. Download doc.pdf.


16. Bridget M, Gebru GN, Odongo GS, Hedberg C, Elduma AH, Kanu JS, Bangura J, Squire JS, Foster MA.Digitalizing disease surveillance: experience from Sierra Leone. Health Policy and Planning [Internet]. 2024 May 30 [cited 2024 Nov 21];czae039. https://doi.org/10.1093/heapol/czae039 PubMed | Google Scholar


17. Soto G, Araujo-Castillo RV, Neyra J, Fernandez M, Leturia C, Mundaca CC, Blazes DL. Challenges in the implementation of an electronic surveillance system in a resource-limited setting: Alerta, in Peru . BMC Proc [Internet]. 2008 Nov 14 [cited 2024 Nov 21];2(S3):S4. https://doi.org/10.1186/1753-6561-2-s3-s4 PubMed | Google Scholar


18. Adokiya MN, Awoonor-Williams JK, Beiersmann C, Müller O.The integrated disease surveillance and response system in northern Ghana: challenges to the core and support functions. BMC Health Serv Res [Internet]. 2015 Jul 28 [cited 2024 Nov 21];15(1):288. https://doi.org/10.1186/s12913-015-0960-7 PubMed | Google Scholar


19. Lubogo M, Karanja MJ, Mdodo R, Elnossery S, Osman AA, Abdi A, Buliva E, Tayyab M, Omar OA, Ahmed MM, Abera SC, Abubakar A, Malik SMMR.Evaluation of the electronic early warning and response network (Ewarn) system in somalia, 2017-2020 . Confl Health [Internet]. 2022 Apr 16 [cited 2024 Nov 21];16(1):18. https://doi.org/10.1186/s13031-022-00450-4 PubMed | Google Scholar


20. Ekzayez A, al-Khalil M, Jasiem M, Al Saleh R, Alzoubi Z, Meagher K, Patel P.COVID-19 response in northwest Syria: innovation and community engagement in a complex conflict . Journal of Public Health [Internet]. 2020 May 21 [cited 2024 Nov 21];42(3):504-9. https://doi.org/10.1093/pubmed/fdaa068 PubMed | Google Scholar


21. Reijn E, Swaan CM, Kretzschmar ME, Van Steenbergen JE. Analysis of timeliness of infectious disease reporting in the Netherlands . BMC Public Health [Internet]. 2011 May 30 [cited 2024 Nov 21];11(1):409. https://doi.org/10.1186/1471-2458-11-409 PubMed | Google Scholar


22. Randriamiarana R, Raminosoa G, Vonjitsara N, Randrianasolo R, Rasamoelina H, Razafimandimby H, Rakotonjanabelo AL, Lepec R, Flachet L, Halm A. Evaluation of the reinforced integrated disease surveillance and response strategy using short message service data transmission in two southern regions of Madagascar, 2014-15 . BMC Health Serv Res [Internet]. 2018 Apr 10 [cited 2024 Nov 21];18(1):265. https://doi.org/10.1186/s12913-018-3081-2 PubMed | Google Scholar


23. Venkatarao E, Patil R, Prasad D, Anasuya A, Samuel R. Monitoring data quality in syndromic surveillance: Learnings from a resource limited setting . J Global Infect Dis [Internet]. 2012 Apr- Jun [cited 2024 Nov 21];4(2):120-7. https://doi.org/10.4103/0974-777x.96778 PubMed | Google Scholar