Modelling the probability and impact of false‐positive serology for Borrelia burgdorferi sensu lato: A case study

Abstract Background Serological screening tests for Lyme borreliosis have poor specificity, with potential for misdiagnosis and unnecessary antimicrobial treatment. Objectives To evaluate the impact of Lyme borreliosis seroprevalence and serologic test characteristics on the probability of obtaining a false‐positive result and impact on antimicrobial use. Study design Cross‐sectional serological survey and modelling. Methods Sera from 303 horses in southern Belgium were analysed by enzyme‐linked immunosorbent assay (ELISA). Apparent seroprevalence was derived from serological data and a Bayesian estimate of true seroprevalence was computed. These were a starting point to model the impact of test and population characteristics on the probability of obtaining false‐positive results and consequently unnecessary treatments and complications. Results Apparent and true seroprevalence were 22% (95% CI 18%‐27%) and 11% (credible interval with 95% probability 0.6%‐21%) respectively. We estimate that two‐thirds of positive samples are false positive in southern Belgium, with one in five of tested horses potentially misdiagnosed as infected. Around 5% of antimicrobial use in equine veterinary practice in Belgium may be attributable to treatment of a false‐positive result. Main limitations There was uncertainty regarding the ELISA's sensitivity and specificity. Conclusions This study highlights the importance of appreciating the poor diagnostic value of ELISA screening for Lyme borreliosis as demonstrated by this case study of seroprevalence in southern Belgium where we demonstrate that a nontrivial number of horses is estimated to receive unwarranted treatment due to poor appreciation of screening test characteristics by practitioners, contributing substantially to unnecessary use of antimicrobials.


| INTRODUC TI ON
Lyme borreliosis is the most prevalent tick-borne infection in the northern hemisphere, affecting both humans and animals. [1][2][3] Reported seroprevalence of B burgdorferi among horses in Europe varies widely from 6% to 36%. [4][5][6][7][8][9][10][11] This variation may be not only caused by geographical differences in seroprevalence but also by the characteristics of the diagnostic tests used to assess prevalence in these studies.
In Europe, enzyme-linked immunosorbent assay (ELISA) and immunofluorescence assay (IFA) are routinely used for serologic confirmation of exposure to B burgdorferi 12 and both tests are currently commonly used in Belgium.
Both tests are considered sensitive but not particularly specific and in both human and veterinary medicine, a two-tier approach to serological diagnosis of exposure to Borrelia spp is recommended whereby a positive screening test such as ELISA is followed by a confirmatory, more specific test, usually an immunoblot 2,12-14 . The first aim of our study was to estimate the seroprevalence of antibodies to B burgdorferi in horses without clinical signs suggestive of Lyme borreliosis in southern Belgium. The second aim was to explore the impact of Lyme borreliosis test characteristics on test interpretation and to estimate the impact of poor test specificity on potential antimicrobial use.

| Survey population
A serological survey was carried out. The population under investigation were all horses housed in southern Belgium which according to the national horse registry (CBC) stood at 73 772 in 2014. All participating horse owners were asked to complete a questionnaire (Data S1). Horses were only included in the study if they were aged at least 12 months and had been housed at their current location for at least the preceding 12 months. Horses were excluded if their owner ticked "yes" on any of the boxes indicating that the horse had, in the preceding 12 months, presented clinical signs that are (or have been in the past) attributed to B burgdorferi infection in horses. The questionnaire also recorded whether horses had pasture access and whether a tick had been found on the horse in the preceding 12 months, but these data were not used in the current study. Samples were either derived from horses presented to the Liege University's equine hospital accompanying hospitalised foals, for elective surgical procedures, traumatic injury, cardiologic evaluation, simple obstructive or strangulating acute colic, or alternatively were collected in the field.

| Serological analysis
Blood was collected via jugular venipuncture into plain serum tubes, allowed to clot, centrifuged and frozen at −80°C until analysis or, for samples collected in the field, stored first at −20°C then within 1 week transferred to −80°C until analysis. Samples were collectively transported frozen to the laboratory (LABEO Frank Duncombe) and analysed by a commercially available ELISA kit (Testkit Borrelia burgdorferi Veterinary ELISA IgG, Virotech Diagnostics).
In a prior study, 16

| Sample size
A power calculation was performed to estimate the number of horses to sample. As no prior data on seroprevalence among horses in southern Belgium were available, we opted to carry out a pre-

| Data analysis
As the ELISA sensitivity and specificity from our preliminary work were markedly poorer than those indicated by the test manufacturer (96% vs 100% for sensitivity and 63% vs 96% for specificity), estimation of true seroprevalence by traditional methods 18 was not considered appropriate due to the uncertainty around the actual test characteristics. As an alternative, we assumed that the actual sensitivity and specificity lay somewhere between our estimate and the manufacturer's. We calculated a Bayesian estimate of true seroprevalence using Markov Chain Monte Carlo simulation 19 with prior estimates of a uniform distribution of 96%-100% for sensitivity and of 63%-96% for specificity.
Additionally, a χ 2 test was applied to compare the expected vs actual proportion of test positive horses in the sample. All statistical analyses were performed in R (R version 3.5.2; R Core Team). 20

| Model scenarios
To explore the potential impact of test performance, we devised the following three scenarios:

| True seroprevalence
Given the test performance as determined in our prior work, the proportion of positive tests which is most likely a false positive was calculated for a range of seroprevalences up to the highest reported in Europe (range 0%-36%).

| Test performance
The effect of varying test specificity and sensitivity between the manufacturer's specifications and our own estimates on the proportion of positive tests that is most likely a false positive was evaluated (sensitivity range 96%-100% and specificity range 63%-96%), with seroprevalence fixed at the point estimate for true seroprevalence from our seroprevalence study.

| Potential overtreatment
The impact on antimicrobial use if every horse with a positive (≥12VE) ELISA result is treated with a 21-day course of antimicrobials is explored. The risk of serious complications following overtreatment was calculated assuming an incidence of 0.6% per course of antimicrobials administered to a horse. 15 In addition, the potential relative contribution of overtreatment on overall antimicrobial use was calculated using local data on average daily doses per horse per year 21 and number of tests submitted annually for Lyme borreliosis screening serology to one regional laboratory. 22 To evaluate whether this overtreatment scenario was likely to happen at all, given the available information in veterinary scientific literature on the pitfalls of Lyme borreliosis diagnosis, we conducted a small survey by issuing a questionnaire (Data S2) to veterinarians visiting Belgian equine (BEPS) or general veterinary (Véterinexpo, AVPL) conferences in the fall of 2019. The Bayesian estimate for true seroprevalence was 11% (credible interval with 95% probability CI 0.6%-21%).

| Impact of true seroprevalence
The expected distribution of true-negative, false-positive, true-positive and false-negative results obtained when test sensitivity and specificity are 83 and 78%, and population true seroprevalence varies, is illustrated in Figure 1. When true prevalence is 11% such as in southern Belgium, then more than two-thirds of all positive samples will be false positives. Therefore, with these test characteristics, any positive serological result in a horse residing in southern Belgium is more likely to be a false positive than a true positive. Predictably, this ratio improves as true seroprevalence increases. However, it is important to realise that even in populations with the highest reported seroprevalence in Europe, almost one in three (32%) of positive results is likely a false positive.

| Impact of test performance
The expected distribution of test true-negative, false-positive, true-positive and false-negative results obtained by fixing true prevalence at 11% and concurrently increasing sensitivity and specificity from 83% to 100% and from 78% to 96% is illustrated in Figure 2. The actual performance of the ELISA kit remains unknown, as outlined previously. The steeper curve towards the right of Figure 2B highlights how especially among higher values, a small deviation from the manufacturer's specifications of specificity has a relatively large impact on the probability of a positive test being a false positive.

| Potential overtreatment
We assume a true seroprevalence of 11%, a test sensitivity and specificity of 83 and 78% (which corresponds to the more conserva-

| D ISCUSS I ON
In the serosurvey part of our study, apparent and true seroprevalence were 22% (95% CI 18%-27%) and 11% (credible interval with 95% probability 0.6%-21%), which falls within the range of reports of 6%-36% in other regions in Europe. [4][5][6][7][8][9][10][11] It should be noted that in many of these studies, performance characteristics of the test used to establish serological status are not discussed and most seem to report apparent seroprevalence. America. 23 So far, reports over its performance and its specificity in particular have varied in studies on North American horses. 23,24 It is likely that at least some of this variation is due to differences in the population they are evaluated in, as perfect specificity was found in a population of pathogen-free ponies, 24 whereas in a patient population specificity was good at 96% but by no means perfect. 23 Crossreactivity with other (spirochete) pathogens could be a possible explanation for the discordance between these estimates of specificity. 23  Our modelling estimates indicate that among healthy horses residing in southern Belgium, around one in five ELISA tests will return a positive result but two-thirds of these will be false positives, due to a combination of poor test specificity and a low true seroprevalence. and it is plausible that those who were not surveyed, may be even more likely to prescribe antimicrobials under a similar scenario.
Our 5% estimate may have overestimated the proportion of veterinarians who would prescribe antimicrobials, but it certainly underestimated the number of serologic tests that were carried out, as Borrelia serology is offered by other regional laboratories besides that from which submission data were available for our study. All in all, considering the above, it is likely that a nontrivial percentage of total equine antimicrobial use per annum has followed a false-positive Borrelia serology. This indicates that addressing the understanding of the relevance of test results for Borreliosis or improving veterinarians' resistance against horse owners demanding treatment after positive serology could be a target for regulators wishing to reduce antimicrobial use. The impact of unwarranted treatment of suspected Lyme borreliosis is relatively high, which is in part due to the prolonged course of antimicrobials that is usually prescribed.
Antimicrobial-associated diarrhoea arising as a complication following unwarranted antimicrobial treatment was estimated to affect only a very small number of tested horses, mostly because incidence of antimicrobial-associated diarrhoea is low in general. However, it is a complication which when it occurs is often costly to treat and has a fatal outcome in almost one in five cases. 15 Overall, a major limitation in most of our estimates is the absence of a gold standard for serological confirmation of exposure to B burgdorferi. We used WB as a reference test but it is possible that some samples were misclassified by this WB. 27   This study did not aim to quantify the impact of incorrectly considering a truly positive Borrelia serology, and therefore evidence of infection, as sufficient proof of Lyme borreliosis. The authors reiterate the advice to follow Consensus recommendations 2 for reaching a diagnosis of Lyme borreliosis before deciding on any treatment.

DATA ACCE SS I B I LIT Y S TATE M E NT
The data that support the findings of this study are available from the corresponding author upon reasonable request.

ACK N OWLED G EM ENTS
The authors thank Ms Simona Cerri for her aid in data collection.

CO N FLI C T O F I NTE R E S T S
C. Meersschaert  Houben analysed the data and prepared the manuscript with input from C. Meersschaert, H. Amory, G. Henrickx and P. Pitel.

E TH I C A L A N I M A L R E S E A RCH
The study protocol was approved by the Liege University's Ethical committee (reference 2015/258).

OWN E R I N FO R M E D CO N S E NT
All owners consented to participation in the study.