Test audience rules with boundary and conflict cases
Prepared with AI assistance by Alactic Editorial. Examples identified as hypothetical are illustrative, not customer results.
An audience count can look plausible while the membership logic is wrong. One contact exactly on a date cutoff may disappear, an unknown permission value may become eligible, or a high score may override an exclusion that was supposed to apply everywhere.
To catch these errors, build a small test set with an expected result for every record before evaluating the audience. Include records immediately before, exactly at and immediately after each boundary. Then add unknown values, conflicting conditions and records that qualify through more than one path. Compare membership by record ID, not just by total count.
The filled fictional pack below shows how to do this without testing automation, message delivery or production execution. Its purpose is narrower: decide whether the audience expression produces the membership the business actually intended.
1. Turn the audience description into a testable contract
Start with the business wording, but do not treat it as a specification. Phrases such as “active in the last month,” “high-value contacts” and “exclude opt-outs” leave important decisions unresolved.
Fictional worked example: a team wants contacts in the North region who have permission recorded and have not opted out. Within that eligible group, a contact can qualify through either recent activity or a high engagement score. All dates, values, records and results in this example are hypothetical.
The team writes the following membership contract:
- Evaluation instant: 1 July 2026 at 12:00:00 UTC, held fixed for the test.
- Required region: the normalized value must equal North.
- Required permission: permission must be explicitly true.
- Required exclusion check: opted_out must be explicitly false.
- Path A: last_activity must fall between 1 June 2026 at 12:00:00 UTC and the evaluation instant, including both endpoints.
- Path B: engagement_score must be at least 80.
- Unknown-value treatment: an unknown required field fails that requirement. An unknown activity timestamp fails Path A; an unknown score fails Path B. One failed path does not prevent the other path from qualifying.
This example uses a rolling 30-day interval, not a calendar-month definition. That choice matters: a calendar-based rule could have different boundaries. Neither interpretation should be left to whoever happens to build the filter.
eligible = region_is_north
AND permission_is_explicitly_true
AND opted_out_is_explicitly_false
recent = cutoff <= last_activity <= evaluation_instant
high_score = engagement_score >= 80
member = eligible AND (recent OR high_score)The parentheses are part of the contract. Every qualifying path must pass the same eligibility requirements.
2. Choose cases that isolate decisions, not typical customers
A useful test pack is deliberately artificial. Most records should differ from a known qualifying record in only one relevant field. That makes a failure explainable: if the only change is one second at the cutoff, investigate the date comparison before investigating the score rule.
Use three groups of cases:
- Boundary cases: immediately inside, exactly on and immediately outside the allowed interval or threshold.
- Unknown-value cases: missing values in both required fields and alternative qualifying paths.
- Conflict cases: a qualifying condition and a disqualifying condition present together, plus records that satisfy multiple paths.
Match boundary precision to the values the evaluator actually retains. If timestamps are stored only to the minute, one-second differences may collapse into the same value. Use the smallest meaningful stored unit and check the values after normalization.
Also separate missing from false. A missing permission value is not an explicit false, but neither meets this example's requirement for explicit true. Likewise, a missing opt-out value is not evidence of an explicit false. Those decisions belong in the contract, not in an undocumented default.
3. Use this filled hypothetical QA test pack
The following pack contains fictional records and invented observations from a deliberately flawed candidate expression. It is not a report of testing an Alactic product. Its observed failures illustrate what a reviewer should be able to diagnose.
Pack header: QA-Audience-v1. Rule contract: v1. Evaluation instant: 2026-07-01 12:00:00 UTC. Lower cutoff: 2026-06-01 12:00:00 UTC. Timestamp precision: seconds. Each test ID represents one distinct contact.
Unless a case overrides a field, use these hypothetical defaults: region North; permission true; opted_out false; last_activity 2026-06-15 12:00:00 UTC; engagement_score 20. “Unknown” means a missing value, not the literal text “unknown.”
ID Change from defaults
Expected | Observed | Assessment
T01 Activity: 2026-06-01 12:00:01 UTC
IN | IN | Pass: just inside lower boundary
T02 Activity: 2026-06-01 11:59:59 UTC
OUT | OUT | Pass: just outside lower boundary
T03 Activity: 2026-06-01 12:00:00 UTC
IN | OUT | Fail: exact lower cutoff omitted
T04 Activity: 2026-07-01 11:59:59 UTC
IN | IN | Pass: just inside upper boundary
T05 Activity: 2026-07-01 12:00:00 UTC
IN | IN | Pass: exact upper cutoff included
T06 Activity: 2026-07-01 12:00:01 UTC
OUT | OUT | Pass: future activity does not qualify
T07 Activity: unknown; score: 79
OUT | OUT | Pass: neither path qualifies
T08 Activity: unknown; score: 80
IN | IN | Pass: score path qualifies alone
T09 Score: unknown
IN | IN | Pass: recent path qualifies alone
T10 Activity: unknown; score: unknown
OUT | OUT | Pass: neither path established
T11 Permission: unknown
OUT | IN | Fail: required permission not established
T12 opted_out: unknown
OUT | IN | Fail: required exclusion check unresolved
T13 opted_out: true; score: 80
OUT | IN | Fail: score path bypasses exclusion
T14 Permission: false; score: 80
OUT | OUT | Pass: permission requirement holds
T15 Region: South; score: 80
OUT | OUT | Pass: region requirement holds
T16 Score: 80
IN | IN | Pass: both paths qualify
T17 opted_out: true
OUT | OUT | Pass: recent path respects exclusion
T18 Region: unknown; score: 80
OUT | OUT | Pass: required region not establishedIn this hypothetical run, 14 of 18 cases match. There is one false exclusion, T03, and three false inclusions, T11–T13. The expected audience contains seven distinct test contacts; the observed audience contains nine.
Those totals are useful as a reconciliation check, but the individual mismatches are the evidence. Two different errors can cancel each other out and leave an apparently correct audience count.
4. Diagnose failures from paired cases
The test pack is most useful when cases can be read together. A single unexpected inclusion tells you something is wrong; a carefully chosen pair helps locate the mistake.
Exact cutoff versus just inside
T01 passes while T03 fails. Both should qualify, but only T03 is exactly on the lower boundary. That points toward a strict comparison where an inclusive comparison was intended. Check the actual stored timestamp and timezone first, then the comparison operator.
Recent path versus score path under exclusion
T17 is correctly excluded, while T13 is incorrectly included. Both have opted_out set to true; the difference is T13's qualifying score. This suggests that the exclusion applies to the recent-activity branch but not to the score branch.
A candidate expression with that problem might effectively behave like this:
region_ok AND permission_ok AND ((recent AND not_opted_out) OR high_score)
Moving an exclusion into one branch changes its scope. The intended structure places all eligibility requirements outside the alternatives:
region_ok AND permission_ok AND not_opted_out AND (recent OR high_score)
Unknown required fields versus unknown path fields
T08 and T09 correctly show that one known qualifying path is sufficient. T11 and T12 expose a different issue: unresolved eligibility fields are being treated as acceptable. Do not “fix” this by rejecting every record with any unknown value. That would incorrectly exclude the valid single-path cases.
These patterns suggest where to inspect the expression; they do not prove the underlying cause. Confirm the diagnosis against the evaluated values and the actual rule grouping.
5. Make expected membership independent of observed membership
Write expected results before running the candidate audience. Otherwise, reviewers may rationalize whatever the tool returns. Ideally, one person translates the business requirement into expected outcomes and another checks the configured expression against that contract.
For every mismatch, preserve a compact evidence record:
- The test ID and rule version.
- The fixed evaluation instant and timezone.
- The relevant stored or normalized field values.
- Expected membership and the reason for it.
- Observed membership and the candidate expression version.
- The proposed correction and the result after retesting.
If a contact is absent from an output, confirm that it was actually evaluated before calling it a logic failure. A missing test record and a correctly evaluated exclusion can look identical in a membership list. Keep that distinction in the evidence, while leaving ingestion and integration testing outside this pack's scope.
If the business cannot agree whether a case should be included, mark it unresolved rather than forcing a pass or fail. Resolve the contract, version it and only then update expected membership. An implementation cannot pass an undefined requirement.
6. Check overlap without confusing membership and row count
T16 qualifies through both paths. Its expected outcome is one audience member, not two. If the available output lists a contact once per matching path, reconcile that output to distinct contact IDs before judging the audience.
Keep two separate observations when possible: whether the contact belongs and which paths it satisfies. Path-level information is helpful for diagnosis, but membership is the acceptance decision. Do not assume that a particular workspace exposes path-level explanations; a reviewer can calculate them separately from the fixture values.
For more complex audiences, test every independent path alone and at least one meaningful overlap. Then test a global exclusion against each path. Testing only the most common route leaves less-used branches free to bypass shared restrictions.
A small deliberate pack is easier to inspect than a large random sample, but it is not exhaustive. If an expression has deeply nested alternatives, simplify the expression where possible and add cases for interactions that carry a real membership risk. Avoid multiplying cases merely to make the pack look comprehensive.
7. Repair the expression, then rerun the whole pack
In the fictional example, the proposed correction has three parts: include the exact lower cutoff, require explicit values for eligibility fields and apply the opt-out check before either qualifying path can admit a contact.
After making those changes, rerun all 18 cases—not just the four that failed. A broad correction to unknown-value handling could fix T11 while breaking T08 or T09. A regrouping change could fix T13 while accidentally making both paths mandatory.
Record the rerun as a new result rather than overwriting the original observations. If the corrected candidate matches the contract, its hypothetical acceptance result would be 18 matching outcomes, seven distinct members, no false inclusions and no false exclusions.
Use a clear acceptance gate:
- Every expected outcome has a documented rationale.
- No membership mismatch remains unexplained.
- Each qualifying path has a passing standalone case.
- Each global exclusion has been tested against the paths it must govern.
- Boundary and unknown-value decisions match the agreed contract.
- The full pack has passed after the latest expression change.
Passing this pack establishes only that the tested audience logic matches its specification. It does not establish message eligibility, template status, delivery readiness or anything about subsequent execution.
8. Leave with a reusable regression pack
Keep the fixture values, expected results and evaluation instant together. Reusing the same records with a moving “now” makes recency cases drift and can turn yesterday's boundary test into today's ordinary exclusion. If the evaluation instant cannot be fixed, regenerate the boundary timestamps relative to one documented reference instant for each run.
Add a new case whenever a real review reveals an untested interpretation: an empty string instead of a missing field, a different timestamp precision, or another qualifying path. Specify how each representation should be normalized rather than assuming all blanks behave alike.
Your next action is concrete: take one audience awaiting acceptance, write its grouped membership expression and build a pack modeled on these 18 cases. Measure agreement by test ID, list every false inclusion and false exclusion, and accept the expression only when the agreed cases match. That gives the reviewer something stronger than a plausible total: an explainable membership decision at the points most likely to go wrong.
Put the next conversation to work.
Explore Alactic Communications for WhatsApp conversations, campaigns and AI-assisted workflows. Choose the product and plan that fit your team.
Explore Communications