Using AI in Machine Vision

Since the term “artificial intelligence” was coined on the campus of Dartmouth College in 1956, it has become common nomenclature across many fields of study – from its origins in philosophy to science, mathematics, and beyond. Despite having existed for decades, the integration of artificial intelligence (AI) into machine vision is relatively new. Today, more and more manufacturers are leveraging the combined power of AI and machine vision to better automate, optimize their efficiency, and improve quality control.

AI augments rule-based machine vision with image-based analysis. When a computer (or vision system) receives an image, AI software compares that image with a database consisting of both “good” and “bad” reference images and outputs a result. At a minimum, the result is pass/fail or OK/NG but can scale in complexity depending on requirements. This process of learning to recognize patterns and infer from annotated reference images allows computers to differentiate between acceptable and unacceptable anomalies in objects under inspection.

Moreover, machine vision solutions embedded with AI technology can use natural language processing to read and interpret labels on images, compared to rule-based approaches which require extensive programming and significant technical expertise. This enables a wider base of users to take advantage of AI for factory automation. Two leading technologies within AI – edge learning and deep learning – help to further simplify automation of highly variable tasks and solve tasks that are too complicated and time-consuming to program with rule-based algorithms.

Edge Learning – Edge learning is a subset of AI in which processing takes place on-device, or “at the edge” of where the data originates, using a pre-trained set of algorithms. The technology is simple to setup, requiring smaller image sets and shorter training and validation periods than traditional deep learning-based solutions.

Deep Learning – Capable of processing large, detailed image sets, deep learning is designed to automate complex or highly customized applications. The technology enables users to analyze vast image sets quickly and efficiently, to detect subtle defects and deliver accurate results.

Edge Learning or Deep Learning

Edge learning and deep learning are both subsets of artificial intelligence (AI). However, there are important differences between these capable technologies, with each having distinct characteristics.

Edge learning differs from deep learning in its emphasis on ease-of-use across all stages of deployment. It requires fewer images to achieve proof of concept, less time for image setup and acquisition, and no specialized programming. Still, each technology has its own unique use cases.

Deep Learning Use Cases

Deep learning simulates the way interconnected neurons in the human brain strengthen and weaken connections to create an understanding of images. In deep learning, hundreds of layers of neural networks are exposed to a large set of images of similar objects. By slightly modifying connections within and between these layers every time it is exposed to a new image, deep learning learns to reliably identify those objects, and detect defects in them, without any explicit training.

Traditional deep learning provides the capacity to process large and highly detailed image sets, making it ideal for complex or highly customized applications. Because such applications introduce significant variation, they demand advanced computational power and robust training capabilities. To account for this variation and capture all potential outcomes, image sets numbering in the hundreds or thousands of images must be used for training. Traditional deep learning enables users to analyze such image sets quickly and efficiently, delivering an effective solution for automating sophisticated tasks. While full-fledged deep learning products and open-source frameworks are well designed to address complex applications, the majority of factory automation applications entail far less complexity, making them better suited for edge learning.

Edge Learning Use Cases

The power of AI can be applied to problems in factory automation by embedding knowledge of application requirements into the neural network connections from the start. This pre-training removes a lot of the computational load, particularly when supported by the appropriate traditional machine vision tools. The result is edge learning, a light and fast set of vision tools.

Edge learning tools can be trained in minutes, using as few as five to ten images. Compare this to deep learning solutions, which can require hours to days of training, using hundreds to thousands of images. By streamlining deployment, edge learning enables manufacturers to ramp quickly, while remaining nimble and able to adjust easily to changes.

In order to optimize the edge learning to run on embedded vision systems, the training images are downscaled or fixtured in a way that only the specific region of interest is analyzed. If these downscaled images were to be differentiated with the line engineer’s own eyes, they can be confident the edge learning tools will perform equally as well. However, it is important to note that this optimization comes at a trade-off. It limits the use of edge learning in very advanced and high-accuracy defect detection applications, which are better solved with traditional deep learning solutions.

How Edge Learning Works

Using a single, smart camera-based solution, edge learning can be deployed on any factory line within minutes. Such a solution integrates several components including machine vision hardware, rule-based tools, and artificial intelligence (AI) capabilities.

Hardware

Edge learning packs sophisticated hardware into a small form factor. It runs entirely in a smart camera equipped with integrated lighting, an autofocus lens, and a powerful sensor. All of these hardware features make edge learning possible.

Lighting is key for a high-quality imaging as it maximizes contrast, minimizes dark areas, and brings out the necessary detail.

A high-speed autofocus lens ensures that the object of interest is always in focus, even as distance changes. It does so by instantly adjusting focus as the region of interest (ROI) changes. Liquid autofocus lenses are smaller and lighter than equivalent mechanical lenses, reducing the size and weight of the camera while making it resistant to the shock and vibration of a production line.

A large and capable sensor offers high resolution and a wide field of view (FOV) for any given application.

Machine Vision Tools

Rule-based vision tools are well-suited for specialized tasks, such as location, measurement, and orientation. For the purposes of edge learning, they are combined in ways specific to the demands of factory automation, eliminating the need to chain vision tools or build complex logic sequences when training the system.

These tools provide fast preprocessing of images, extracting density, edge, and other feature information for the purposes of detecting and analyzing manufacturing defects. By identifying and clarifying the relevant parts of the image, these tools reduce computational load, compared to traditional deep learning approaches.

AI Capabilities

Instead of using rules created by human programmers, AI learns by example, building a neural network and devising effective pass/fail thresholds from labeled examples of acceptable and unacceptable parts. In essence, it mimics the way humans learn.

AI capabilities can have large training requirements. Edge learning, on the other hand, takes advantage of the fact that factory automation images have specific structural contents, and so pre-trains its algorithms with that domain knowledge. Not starting from scratch results in a less programming-intensive application.

Edge Learning Application Examples

With simple setup and the ability to solve tasks of varying complexity, edge learning is a viable solution for deploying automation across a wide range of industries. This capable technology brings the power of artificial intelligence to all, enabling users of varying experience and skill levels to run reliable inspections.

Automotive

In vehicles, connectors are used to form a continuous electronic signal by connecting wire harnesses to a power source or circuit within vehicles. In the simplest terms, connectors consist of jacks that are joined together to relay an electronic signal. A male-ended plastic housing fits into a snap-mounted female-ended chassis mounting. Because the pair can be connected or disconnected easily, it is important to verify full insertion, which ensures proper functioning of electrical components within the vehicle.

Edge learning can determine whether the jacks are correctly seated. In addition, it can be retrained to classify connectors by different shapes (circular, header, pin, crimped, multitap), materials, quality, and dimensions depending on the application.

Electronics

Many printed circuit boards (PCBs) include LED indicator lights to show status. Typically, the status is one of three conditions: power on (PWR), transmit (TX), or off (OFF).

Using traditional machine vision, the standard way to determine whether indicators show a PWR, TX, or OFF condition is with a pixel count tool. This involves setting thresholds for brightness at specific locations for each condition, a complex process that requires extensive programming expertise.

Edge learning, however, can be trained on a small set of labeled images of the PWR, TX, and OFF conditions. After this brief training, edge learning reliably sorts the PCBs into the three different states.

Life Sciences

Test tubes are commonplace in lab diagnostic environments. A single tube can convey a considerable amount of information – from a barcode adhered to the side to the color of the cap – that helps clinicians to properly process test samples. Cap color, in particular, is useful in distinguishing one sample from the next. For instance, the cap color often denotes what additives have been mixed with the sample to stabilize and preserve the specimen prior to testing.

Edge learning can not only detect the presence/absence of a cap on a test tube to ensure it’s properly sealed, but can go one step further and identify the color of the cap. This enables clinicians to more efficiently run lab diagnostics and ensure samples are correctly processed.

Medical/Pharmaceutical

In the medical and pharmaceutical field, glass vials are often filled with medication to a predetermined level. Before they are capped, the fill level must be confirmed to be within proper tolerances. The transparent and reflective nature of both the glass vial and its contents makes it difficult for traditional machine vision to consistently detect the fill level.

Edge learning can be deployed to identify the level, without getting confused by reflections, refraction, or other disorienting variables within the inspection image. Fills that are too high or too low are classified as NG (failing), while only those within the proper tolerances are classified as OK (passing).

Packaging

Before distribution, bottles of soft drinks and juices are filled and sealed with a screw cap. If the rotary capper miss reads the cap, or it gets damaged during the capping process, this can leave a gap that allows for contamination or leakage. Both the speed and the wide range of ways in which a cap may be almost, but not quite sealed, make this a challenging application for traditional machine vision to automate.

Edge learning can be given a set of images labeled as either good (properly sealed caps) or bad (caps with slight, almost imperceptible, gaps). While maintaining line speeds, edge learning identifies fully sealed caps and categorizes them as OK. All other caps that do not meet the quality threshold are classified as NG.

Acrovision OCR Applications

1.  24 laser etched dot code characters on reflective foil read at high speed (500 parts per minute)

2. Validating hot stamp codes on lipstick labels. Demonstrates how Edge Learning OCR is tackling applications traditional OCR tools cannot solve.

3. Validating dot peen product codes on exhaust systems.