On an inspection station along a production line, four industrial cameras capture every metal part moving down the conveyor. Each frame has to be judged for defects, and that inference typically has only a few tens of milliseconds of window. A round trip over the public internet usually costs over 100 ms. In this scenario, compute has to sit as close as possible to where the data is generated — edge AI deployment, often referred to as local deployment.

When specifying AI for production-line intelligence, the hard part isn't the technology. It's deciding when cloud is sufficient and when inference has to move on-premises. Cloud offers more compute, lower unit cost, and no maintenance burden — but it comes with site-level prerequisites.

Four questions decide where compute belongs: can the network be reliably covered? Can raw data leave the site? What is the response-time requirement? And how large will the call volume grow?

丨Where the network drops, inference can't live in the cloud

Production floors rarely guarantee persistent connectivity. Metalworking shops are a textbook case: overhead cranes and servo drives operating simultaneously cause significant wireless attenuation, and short outages — from a few hundred milliseconds to tens of seconds — are routine.

Cloud inference depends on the network for the entire path: images are uploaded to a data center hundreds of kilometers away, processed, and returned. If any segment of that path fails, the entire inference fails.

On a production line, such a failure manifests as a stopped line, a mis-actuated robotic arm, or a missed security check — with direct physical or financial consequences. The workshop has a network, but it drops; the decision hinges on the frequency and duration of those drops. There is also a class of sites — autonomous haul trucks in mining, automated lifting in ports — where the working area is too large for stable coverage, and compute must be deployed locally.

丨If raw data can't leave the plant, inference stays local

Images captured on a line carry process information: fixture geometry, machining parameters, batch identifiers. Defect morphology and location often map back to the parameters of a specific process step. Uploading such images to a public cloud means the raw data leaves the enterprise's control.

The nature of the data determines whether local deployment is required. If what's uploaded is a desensitized result — a defect class label, a pass/fail verdict — cloud processing is fine. Once inference requires uploading raw material, such as full-resolution images or multi-frame sequences, inference has to run locally.

Beyond that, original medical imaging studies, financial identity-verification materials, and government facial-capture data typically carry explicit localization requirements.

There's also a case that isn't about compliance at all, where full upload simply doesn't pay off. A local inference pass can pre-filter on-site video and upload only anomalous segments and structured results to the cloud. Upload volume drops sharply, and bandwidth and cloud storage costs fall with it. Local deployment isn't mandatory here — but it's usually the more economical choice.

丨Tens of milliseconds of latency: the round trip is the bottleneck

Many teams attribute latency to compute, assuming a more powerful cloud GPU must mean faster inference. In reality, GPU compute accounts for only a fraction of end-to-end time. Most of it is spent in transit.

A cloud vision inference consists of several stages: image encoding and compression, upload queuing, cloud inference, and result return. Local vision inference differs in link structure — capture and compute complete within the same device, images don't need to be encoded, compressed, and sent over the public internet, and local inference can land within a few tens of milliseconds.

For latency-critical applications — robotic arm collision avoidance, AGV emergency stops, closed-loop robot control — response requirements often sit under 10–20 ms, and compute has to be deployed locally.

丨As call volume scales, cost stops being predictable

The cost risk isn't in the unit price. It's in the coupling to call volume. Cloud bills by invocation or by token: a single call is cheap, but cumulative spend grows linearly with usage. Local deployment is a one-time investment, and for high-frequency, steady workloads the marginal cost approaches zero.

At the POC stage, with low volume and a single scenario, cloud is the fastest way to start. At scale, with volume up sharply, local unit inference cost is lower.

So the question of cloud versus local comes down to these four: can the network be reliably covered? Can raw data leave the site? What is the latency requirement in milliseconds? How large will call volume grow? If the answer to either of the first two is no, local deployment should be on the table.

Local deployment: start the selection with memory

Once inference is moving on-site, the first decision is how large a model to run.

Whether a model fits depends on parameter count and quantization precision, not on compute. At 4-bit quantization, every billion parameters occupies roughly 0.5 GB — a 7B model takes about 3.5 GB. But fitting isn't the same as running: the OS, image buffers, and intermediate results all consume memory. So the real question is how much memory can actually be allocated to the model.

If you're evaluating local AI deployment or private LLM deployment and need selection guidance, contact PLink-AI.