Our crash reports came from eleven hundred device models. Four of them carried a quarter of our users. Six hundred had fewer than ten users each.
We were testing against a grid that bore no relationship to that distribution. Here is how our Android App Development team decided what to support and what to let go.
The long tail is real and it is not free
The arithmetic that forces a decision.
Every supported model multiplies the surface area. So does every operating system version, screen size, and manufacturer customisation. Most of that surface serves almost nobody.
Our six hundred rare models represented about two per cent of users. They took a far larger share of our defect investigation time. Rare devices produce unusual problems.
Decide a support policy and publish it
Internally and to customers.
Ours states the minimum operating system version, the memory floor, and the screen size range we test against. It carries the date it was last reviewed.
Before this, support was whatever the newest developer assumed. Two features shipped with a minimum requirement nobody had agreed, and support discovered it from complaints.
Weight by users, not by models
The reframing that made the choice obvious.
Eleven hundred models is an intimidating number. Covering ninety four per cent of our users takes about thirty devices, chosen by actual distribution.
Android App Development teams that think in model counts conclude the problem is intractable. Thinking in user counts makes it a straightforward selection.
Keep the oldest supported device physically present
Not an emulator.
Our minimum spec device sits on a desk. Every significant feature is run on it before release.
An emulator will not show you that a screen takes nine seconds to open on real hardware with real storage pressure. Ours did, and that defect would otherwise have reached about eleven per cent of our users.
Separate the operating system version question from the device question
They move differently.
Our version distribution shifts steadily as devices update. Our device distribution shifts slowly, because people keep phones for years in several of our markets.
We review the version floor every six months and the device grid annually. Treating them as one decision produced a policy that was wrong about both.
Watch what the manufacturers change
The variable that surprises teams.
Manufacturer customisations differ substantially. Background execution, notification delivery, and battery optimisation all vary. None of it is fully documented.
Three of our worst defects were in this category and none of them appeared on a reference device. Our grid now deliberately includes the manufacturers our users actually hold, not the ones with the cleanest software.
Make the front end consistent across platforms
Where the duplication lives.
Our web and mobile surfaces had drifted apart. Different interaction patterns, different terminology, and different validation messages for the same operations.
Our Front-End Services work produced a shared design system and shared validation rules. The Android implementation consumes those rather than reimplementing them, which removed a recurring category of inconsistency complaints.
Be disciplined about experimental features
Our approach to an uncertain addition.
We ran a generative AI proof of concept for in app support, summarising a user's recent activity to pre fill a support request.
It worked on recent devices. On our minimum spec hardware it performed poorly. That is exactly the user most likely to need support. We shipped it to devices above a threshold. The manual path stayed in place everywhere. A promising feature should not quietly raise your minimum requirement.
Instrument before you cut
So the decision has evidence.
Before removing support for anything, we know how many users it affects, what they would experience, and whether a usable alternative exists.
We removed two operating system versions last year. Both were under one per cent and both removals were announced in the app three releases ahead.
Review the policy on a schedule
Because distributions move.
Ours is reviewed twice a year against current analytics. Last review raised the version floor by one and added two manufacturers to the grid.
Without the schedule, the policy ages into something written for a user base that no longer exists, which is how we ended up with the eleven hundred model problem.
What our support policy specifies
- A published minimum operating system version, memory floor, and screen range.
- A test grid of about thirty devices chosen by user distribution.
- The minimum spec device physically present and used before release.
- Version floor reviewed twice a year, device grid annually.
- Manufacturers chosen by what users hold, not by software quality.
- Shared design system and validation rules across surfaces.
- Capability thresholds for experimental features, with manual fallback.
- Three releases of notice before removing support for anything.
Common Questions
How many devices should be tested?
About thirty covers ninety four per cent of our users. Pick by distribution rather than by trying to cover the catalogue.
Are emulators sufficient?
For most functional testing, yes. For performance and storage behaviour on older hardware, no, and that is where our worst defects were.
How low should the minimum version go?
Where the user count justifies the cost. Ours is reviewed twice a year and has risen once in two years.
Does cross platform consistency matter?
To users, considerably. Our Front-End Services work on shared rules removed a complaint category entirely.
Who decides the support policy?
Product and engineering together. Android App Development teams alone will set it by what is convenient to build.
What is the most common mistake?
Testing the newest devices. Our users hold older ones. Android App Development grids drift towards whatever handsets the team was given.
How do you announce a change?
In the app, three releases ahead. Nobody has complained yet.
How do you handle experimental features?
Threshold them and keep the fallback. Android App Development teams can otherwise raise the effective minimum spec without anybody deciding to.
Where we ended up
A published policy, thirty devices chosen by distribution, the oldest supported handset on a desk, and two version removals announced three releases ahead.
We support fewer devices than we did and fewer users are affected by defects, because the testing now matches what people actually hold rather than what the catalogue contains.
