Computer Vision · Quality control
LLMs in Visual Inspection and Quality Control
Can language models with image understanding detect defects reliably? We tested GPT-4o and Gemini 2.0 Pro on circuit boards and on images from industrial inspections. They can help where conditions vary, but they do not reach the reliability of specialised methods.
Aleksandra Osztynowicz
Key takeaways
- On circuit boards, GPT-4o reached an F1 score of at most 0.68, while traditional anomaly detection achieves an AU PRC of 0.98.
- In industrial inspection, accuracy on the question “defect or no defect” was 0.73 for GPT-4o and 0.66 for Gemini.
- Via an API, analysis takes 1 to 10 seconds per image, too slow for real-time quality control.
- LLMs work as a complement where conditions vary a lot. Human review is still required.
Can LLMs match specialised methods?
Large language models (LLMs) are advancing quickly and opening up new options for visual inspection. For image data, the more accurate term is large multimodal models (LMMs), because they process images or audio as well as language. We stick with the more familiar term LLM here.
Many companies want to test LLMs for their own quality control. The key question is whether they can match methods built specifically for the task. To find out, we ran two experiments under real conditions. The first dealt with defects on circuit boards, the second with industrial inspections, where conditions and defects vary widely.
The tests date from March 2025 and use the leading models of that time. Newer models may perform differently, and the methodology carries over.
Test 1: defect detection on circuit boards
We used the VisA dataset (PCB4) from AWS Open Data. It is demanding: there are few defect images, the classes are imbalanced, and the task is multi-label classification. Some defect types are common, others barely appear, which makes it hard for any model to generalise across all defects.
GPT-4o received an image along with a description of the possible defect types. The model had to name the defect or confirm that the board was defect-free.
We tested 40 images in two scenarios: defective images only, and a mix with defect-free ones. With one-shot learning, the model saw a reference image of possible defects before the test.
| F1 score | No example | One-shot |
|---|---|---|
| Defective images only | 0.37 | 0.51 |
| Defective and defect-free | 0.59 | 0.68 |
The F1 score measures how well a model handles imbalanced classes; the closer to 1, the better. GPT-4o recognised some defects reliably and missed others entirely. The higher score in the mixed scenario came mainly from correctly classified defect-free boards. Traditional anomaly detection methods have reached a comparable AU PRC of 0.98 in prior studies.
Source: Zou, Jeong, Pemula, Zhang and Dabeer, SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation, arXiv 2022
Test 2: defect detection in industrial inspection
Circuit boards are usually inspected under controlled conditions. In industrial inspections, lighting, object orientation and image resolution vary a lot. For the second test we therefore used images and videos from a client’s dataset. Visually similar defect types, several labels per image and sparse labelling added to the difficulty.
We tested ChatGPT-4o and Gemini-2.0-pro-exp-02-05, at the time the leaders of the Chatbot Arena for vision tasks. In the first approach, the models had to name the most likely and the second most likely defect. One-shot examples improved the predictions but did not remove the need for manual review by a human operator. The second approach reduced the task to a binary decision: defect or no defect.
| Accuracy | First test | Defect yes/no |
|---|---|---|
| ChatGPT-4o | 0.61 | 0.73 |
| Gemini 2.0 Pro | 0.63 | 0.66 |
The simpler question produced better scores. For a production-grade alarm system that is still not enough, because industrial systems demand near-flawless reliability.
The main hurdles in industrial use
- Processing time: via an API, each image is processed individually. Depending on the request and the number of one-shot examples, that takes 1 to 10 seconds per image. On fast production lines, that is too slow for real-time quality control.
- Hardware and scalability: smaller, locally hosted models such as Qwen or Molmo reduce reliance on cloud APIs but require substantial hardware investment. Equipping several inspection sites adds logistical and infrastructure effort.
Our article on custom LLMs explains why models tailored to a company are worth considering.
Conclusion: a complement, not yet a replacement
LLMs keep improving, but they are not yet a standalone solution for industrial visual inspection. They can support individual tasks and need human oversight and additional infrastructure to do so. In controlled environments, specialised machine learning models remain superior.
The picture changes away from the standard production line. Where conditions vary, images differ and data is captured on mobile devices, LLMs can be a useful additional tool.
Only a test under real conditions shows which approach holds up in a given case. We assess AI models for practical use, compare LLM-based, traditional and hybrid approaches, and support integration into existing workflows so the solution runs reliably and at scale.
AI in your quality control?
We test on your images whether an LLM, a traditional model or a combination detects your defects reliably.
Request a testFrequently asked questions
No. LLMs can help detect defects and anomalies, but they do not yet reach the reliability of specialised machine learning models or rule-based systems. That applies especially in industry, where consistent results are critical.
Processing times of 1 to 10 seconds per image via an API, dependence on cloud services, varying image conditions, and inconsistent results that require human validation.
In exploratory analysis, under highly variable conditions and when labelled data is scarce. In well-structured production processes, traditional models are more accurate and reliable.
Through structured experiments on their own images and a comparison with traditional methods. That shows which approach is reliable enough for the specific application.
Aleksandra has been building custom AI solutions at theBlue.ai since 2021, with a focus on agentic implementations and local LLM deployments that precisely fit enterprise needs. As an AI Engineer, she helps organizations automate their processes with production-grade systems, from on-premise open-source models to RAG-based internal knowledge bases for regulated industries, and continuously expands her knowledge in the rapidly changing world of artificial intelligence.
In her articles, she shares practical experience from real enterprise AI projects and shows that deploying AI in companies doesn’t need to be complicated.