evaluating antibiotic effectiveness

A predefined number, M, of additional successful outcomes for one drug over another serves as the stopping rule. The antibiotics are tested in vitro using repeated assays to assess whether cell growth is inhibited. The test stops once the difference in the total number of successful inhibition tests reaches M, determining the better drug. However, due to various variability, a less effective drug might sometimes appear superior. This option calculates the error probability of reaching a wrong conclusion.

Explaining the Stopping Rule in evaluating Antibiotic effectiveness with an Example

1. Scenario Setup

Suppose we are testing two antibiotics:

  • Antibiotic A (success rate P1=0.6)
  • Antibiotic B (success rate P2=0.4)

A success means that the antibiotic inhibits bacterial growth effectively.

We decide to stop the test early if Antibiotic A achieves at least m=5 more successful treatments than Antibiotic B.

2. Trial Process

We test these antibiotics on multiple bacterial samples in a randomized experiment. Let's assume that we conduct tests in sequential batches, recording whether each test is successful (growth inhibition) or unsuccessful.

Trial Data Example

Test No. Antibiotic A Antibiotic B Cumulative Success A Cumulative Success B Difference (A-B)
1 ✅ Success ❌ Failure 1 0 +1
2 ❌ Failure ✅ Success 1 1 0
3 ✅ Success ❌ Failure 2 1 +1
4 ✅ Success ✅ Success 3 2 +1
5 ✅ Success ❌ Failure 4 2 +2
6 ❌ Failure ❌ Failure 4 2 +2
7 ✅ Success ❌ Failure 5 2 +3
8 ✅ Success ✅ Success 6 3 +3
9 ✅ Success ❌ Failure 7 3 +4
10 ✅ Success ❌ Failure 8 3 +5 (Stop test)

After 10 tests, Antibiotic A has achieved 5 more successes than Antibiotic B, so we stop the test and conclude that Antibiotic A is more effective.  In this setting the error probability identifying the antibiotic B is superior than A is 0.017.

3. The Problem of Randomness

Even though Antibiotic A is truly better (since its actual success rate is 60% vs. 40% for B), random variations in the test process can sometimes produce misleading results.

For example, let’s say the true probabilities were reversed, and Antibiotic B was actually better, but just by random chance, it started off stronger in the first few tests. The stopping rule might erroneously favor the inferior drug before we collect enough data to be confident.

Example of a Misleading Scenario

Test No. Antibiotic A Antibiotic B Cumulative Success A Cumulative Success B Difference (A-B)
1 ❌ Failure ❌ Failure 0 0 0
2 ❌ Failure ✅ Success 0 1 -1
3 ✅ Success ✅ Success 1 2 -1
4 ❌ Failure ✅ Success 1 3 -2
5 ✅ Success ❌ Failure 2 3 -1
6 ❌ Failure ✅ Success 2 4 -2
7 ❌ Failure ✅ Success 2 5 -3
8 ✅ Success ✅ Success 3 6 -3
9 ❌ Failure ✅ Success 3 7 -4
10 ❌ Failure ✅ Success 3 8 -5 (Stop test)

Here, Antibiotic A was actually worse, but the stopping rule triggered and favoring the wrong drug is identified as a superior drug. In this setting the error probability identifying the antibiotic B is superior than A is 0.017.

a Markov Model

A Markov model can estimate the total test number and the probability that, due to random chance, we select the less effective antibiotic as a superior antibiotc.