Our client is an OEM in the inspection industry and they manufacture high-speed sorting and grading machines for industrial applications. The end users of the sorting machines expect very high throughput with very few sorting defects (Both false positives and false negatives). The cycle time available for completing the process starting from sorting decision to actuation of pneumatic ejectors is 3 ms. The microcontrollers in the current generation machines have the processing capacity only to perform simple thresholding techniques in such short cycle times thus restricting the quality of inspection possible in these machines. The problem we had to solve was to improve the image processing capabilities of the Sorting Machines within the same cycle time.
Solution Design
The image capture part of the sorting machine is a controlled environment with well-designed background and illumination that ensures proper visibility of the object. As this application is high speed and requires high resolution capture the solution was designed with Line Scan Sensors. The output of the sensor returns 2048 pixels for every line scan at the speed of 10 Khz. One of the key challenges was the selection of an appropriate processor that has the capacity to make a sorting decision by running vision algorithms and sending a rejection signal within a cycle time of 3 ms.
Considering we had to enhance the capacity of the processor to run advanced vision algorithms we redesigned the solution using FPGA Zynq Processor. We developed the vision algorithms in C++ and used Vivado HLS tools to synthesise into HDL.
Image Processing Algorithm
The image processing algorithms had to be enhanced from simple thresholding to spectral analysis on the pixel intensities.
Thresholding using ARM based Microcontrollers
The current generation sorting machines manufactured by our client performed very simple thresholding techniques to decide if the object is defective. The pixel intensity of the grayscale input from the line scan camera module was compared with a threshold pixel intensity value and then the sorting decision was made.
Some of the drawbacks of this method were –
- It can be deployed only for a single color object that can be converted to gray scale for implementation of thresholding techniques
- Size, shape or form based defects cannot be identified using this process
- Sorting decision was made using partial image of the object making it difficult to manage the ejection timing
Histogram Spectral Analysis using FPGA processor
We have designed an image processing algorithm using OpenCV that aggregates the pixels of a complete object and plots the pixel intensity on a histogram.
We have designed an image processing algorithm using OpenCV that aggregates the pixels of a complete object and plots the pixel intensity on a histogram.
The sorting decision is made basis the distribution of the pixel intensity of the object. In this case the object is rejected if there is no single peak and the pixel intensity distribution has more than one peak or if there is significant count of pixels across intensity values.
The inherent parallelism in image processing is exploited using FPGA processors to complete the enhanced image processing algorithm within the same cycle time.
Conclusion
We enhanced the solution to handle 50 K + sorting decisions / sec with improved defect identification capability using Spectral Analysis. The sorting machines are now capable of handling objects with more than one colour. Since the processor is upgraded to FPGAs the image processing algorithms can be enhanced in the future to process size, shape and form based defects.