Why ‘It Worked in the Pilot’ Is the Most Dangerous Sentence in IoT
Most industrial IoT projects die in the gap between a clean pilot and a fleet of 50,000 devices. The failure is rarely a worse idea at scale – it is the scale-relevant decisions nobody made while the pilot was still cheap to redesign.
The numbers tell the story before the label does. According to McKinsey, roughly 70% of manufacturers running industrial IoT projects cannot get past the pilot stage, blocked by cost, cybersecurity, interoperability and installation issues. Bain & Company adds another layer: even among companies actively experimenting with IoT, about 80% manage to scale fewer than 60% of their pilots into full production.
‘Pilot purgatory’ is a fair name for this pattern, but naming it does not explain it. It describes the symptom, not the cause. The more useful question is narrower: what specifically breaks when you go from 50 devices to 50,000? That is what we get into below.
1. Hardware that passed the bench dies in the field
A pilot is, almost by definition, deployed somewhere convenient and run under conditions that do not stress the system. That is precisely why hardware limits stay invisible until rollout: the environment that would expose them is the one thing a pilot is designed to avoid.
Three mechanisms explain most of what goes wrong, and none of them show up on a short bench test.
RF propagation almost never matches the datasheet
Range figures assume free space, but a real facility means reinforced concrete, steel racking and metal enclosures acting as partial Faraday cages. A 2.4 GHz link that is clean on a bench can drop out behind two interior walls, while sub-GHz bands push through structure better but sacrifice bandwidth for it. The only way to know in advance is a site survey logging actual RSSI and SNR at each planned device location; there is no formula that substitutes for this.
Power budgets are set by the radio and eroded by temperature
A transmit burst pulls far more current than idle, so duty cycle, not battery size, ends up governing field lifetime. Chemistry compounds this: lithium-thionyl-chloride cells handle wide temperature swings but sag under sudden pulse loads, while standard Li-ion loses capacity in cold and should not be charged below 0°C. A battery rated for five years at 25°C can fall well short of that in an unheated site transmitting on every alarm event.
Environmental stress is cumulative
Moisture ingress, corrosion and electrical shorts are the recurring causes of field failure in outdoor electronics, which is why field-grade sensors and EV chargers depend on conformal coating (parylene, acrylic, silicone or urethane) and properly rated enclosures, and why DC fast chargers are built to shut down on cooling or thermal faults. Heat shortens component life along an Arrhenius curve; vibration fatigues solder joints over months. None of this has time to appear during a pilot.
Put these three together and you get the failure mode that actually costs companies money: a dashboard reporting ‘available’ while the fleet’s real success rate tells a different story. ChargerHelp’s data is a clean illustration – a green status and a 71% real success rate describe the same fleet, but they are not the same picture. That gap is what a pilot is most likely to miss, precisely because a pilot is where the green number looks most convincing.
The practical implication is not about location; it is about instrumentation. Deploy the pilot in the worst site available, and instrument those units to log RSSI, battery voltage under load, internal temperature and reset counts.
2. Fleet economics invert between pilot and production
In a pilot, cost is often not even measured, because the work that generates it is invisible. Fifty devices in one facility get reflashed over a lunch break, watched on a single dashboard, and swapped out by the same engineer who installed them. None of that is a process. It is a person doing something by hand, and it disappears the moment the fleet is too large for any one person to touch. Three costs specifically cross that line, usually after the hardware has already shipped.
Firmware updates are the clearest case
Pushing a build over USB or a local network works fine for a handful of units and says nothing about what is needed at scale. Production requires over-the-air updates with three properties a pilot has no reason to test:
- Staged rollout. Canary to a small percentage of the fleet, watch crash and connectivity telemetry, then ramp. Push a bad image to everyone at once, and it is a fleet-wide incident, not a bug ticket.
- Atomic install with rollback. Usually an A/B partition scheme, so a failed update boots back to the last known-good state instead of bricking a device nobody can physically reach.
- Delta updates. Shipping only changed bytes, because a full multi-megabyte image over a metered cellular connection, multiplied by tens of thousands of devices, turns into a cost line finance will ask about.
Connectivity is a recurring cost with a decade-long tail
It looks like a one-time decision and is not. The 2G/3G sunset makes this concrete. As of December 2025, GSA (the Global mobile Suppliers Association) had tracked over 300 completed, planned or in-progress legacy-network shutdowns across 88 countries. In the US, T-Mobile postponed its 2G shutdown twice before beginning a phased retirement in February 2025.
Fleets provisioned on cheap 2G modules years ago are now stranded. France’s national elevator federation has publicly asked regulators to delay the shutdown, citing roughly 232,000 elevators still running on 2G – each one a safety-regulated device requiring a working emergency link.
The radio choice made at deployment is a bet on network lifespan:
| Option | Strength | Trade-off |
|---|---|---|
| NB-IoT | Cheap; penetrates concrete well | Cannot handle large OTA payloads or mobility |
| LTE-M | Supports large payloads and mobility | Costs more |
| Carrier-managed eSIM (GSMA SGP.32) | Turns a future network switch into a software action, not a truck roll | Requires remote-provisioning support in the module and platform |
Migrations forced by a shutdown deadline are reported across the industry as substantially more expensive than planned ones, since years of lead time get compressed into months. The specific premium varies too much by fleet and region to cite a number with confidence, but the direction is not in dispute: model the connectivity lifecycle before the pilot, not after a carrier sends notice.
Support cost follows the same logic
It shows up most clearly in public EV charging. ChargerHelp’s 2025 reliability report, drawn from over 100,000 charging sessions across 2,400 chargers, found reported network uptime above 98%, yet only about 71% of charging attempts succeed on the first try. Success rates near 85% on new stations fall to roughly 70% by year three. Nearly a third of attempts fail on hardware the dashboard calls available, and a pilot fleet will not reveal this because pilot devices get individual attention that production devices never receive.
Scale requires per-device identity, typically X.509 certificates provisioned at manufacture, a way to rotate or revoke credentials without a truck roll, and a real budget line for the support headcount this tail demands.
The test before any of this ships: build the cost-per-device-per-year model, covering connectivity, device management and support, and check whether the per-unit value still holds at a thousand times the pilot’s device count.
3. Integration is the long pole, and pilots route around it
A pilot can stream clean sensor data to a dashboard and look finished while skipping the part of the job that actually determines whether it scales: making that data usable inside systems the site already runs. Industrial sites are brownfield by default. PLCs, SCADA, historians, an MES, an ERP and decades of legacy serial equipment are already there, and a new IoT layer has to speak to all of it, not replace it.
The integration work is protocol translation
Production data does not arrive as a tidy REST API call. It arrives over Modbus, PROFINET, EtherNet/IP and legacy serial links, each with its own conventions. The current centre of gravity for bridging that gap is OPC UA. Current brownfield reference architectures model proprietary controllers, such as the Siemens SIMATIC S7-1500, into a standardised address space through edge gateways, and semiconductor fabs in particular are standardising on OPC UA at scale, with equipment from Applied Materials, Lam Research, Tokyo Electron and ASML increasingly shipping native support. Vendors in this space commonly cite 50 to 70% lower integration cost compared with building custom per-tool interfaces, which is precisely the cost line that stalls a pilot if it is left unaddressed.
A register value is not data until it is labelled
A raw read like 40001 = 72 carries no meaning on its own; it needs the asset it belongs to, the engineering unit and the scaling factor applied to it before anyone can use it. That gap is what the Unified Namespace pattern solves: a single MQTT-brokered, context-rich source of truth that OT and IT systems both publish to and subscribe from, typically structured with Sparkplug B birth/death certificates over an ISA-95 hierarchy. Practitioners now actively warn against the alternative, piping raw OT data straight into a cloud data lake. It looks fast during a pilot and produces expensive, low-reuse sprawl once the fleet scales. The decision of where contextualisation happens – at the edge, in the broker, or downstream – needs to be made deliberately, before data volume makes the decision by default.
The IT/OT boundary is a security choice
Connecting operational equipment directly to the cloud collapses network segmentation that exists for a reason. A production architecture preserves that boundary, using an edge gateway, a DMZ and controlled northbound flows along Purdue-model lines, so that adding a sensor does not quietly put a PLC on the internet. Retrofitting that separation after a flat pilot architecture has already shipped costs far more than designing it in from the start.
What to decide before the pilot
A successful pilot and a deployment that survives contact with 50,000 units answer two different questions, and most projects fail in the space between them – not because they got worse ideas at scale, but because nobody decided the scale-relevant things while the pilot was still cheap to redesign. That is the actual point of the exercise: use the pilot to force answers to the questions that only matter at fleet size, before the fleet exists.
1. Decide what a device actually costs over its life
Do not extrapolate unit economics from a handful of free pilot units; model them at something close to full fleet size instead. That means pricing in connectivity’s multi-year lifecycle and sunset exposure, device management, and the support headcount that the problem tail will eventually demand. Extrapolation from ten units hides exactly the costs that matter at ten thousand.
2. Decide which signal you will actually trust
An uptime dashboard and first-time-success telemetry can tell two different stories about the same fleet, and only one of them is real. Run the pilot in the worst environment available, instrument it for failure data, and commit now to trusting the harder number later. The EV-charging gap between ‘99% uptime’ and a 71% real success rate is what happens when that decision gets made by default instead of on purpose.
3. Decide how data will actually reach the rest of the business
A sensor that streams clean numbers to a dashboard has not proven anything about integration. Before scaling, settle how real field protocols get translated into the data layer, where tags get contextualised before they reach enterprise systems, and how the IT/OT boundary gets respected, while all three are still cheap to change rather than expensive to retrofit.
If this modelling needs to happen before capital is committed, working with an IoT development company that has already taken fleets past the 50,000-device mark can shortcut a lot of the trial-and-error above.
Figures cited (McKinsey, Bain & Company, GSA, ChargerHelp) are as supplied by the author. Statistics reflect the sources current at the time of writing.