FPGA implementation of real-time skin color detection with mean-based surface flattening

ABSTRACT

Skin color is widely used in many applications because of its merit in human-machine interactions. However, detecting skin color requires repetitive operations on all pixels in the image, similar to other vision-based applications. Since the per-pixel processing is difficult to perform efficiently in conventional computers, many real-time image processing applications have problems with performance. In this paper, we propose FPGA implementation of a real-time skin color detection system. Among the various skin color detection methods, we chose a parametric skin distribution modeling method based on a Gaussian mixture, due to its acceptable training amount and skin detection performance. In addition, a mean-based surface flattening method was also proposed and implemented to improve the detection performance. The proposed method flattens the surface of objects in the scene by replacing the pixel value with the mean of its similar neighborhoods to remove the color noise. After this flattening process, the pixel values of the analogous adjacent pixels are located within a narrow range and are easily segmented to a different region. To consider the inherent parallelism of local image processing, all these functions are implemented within the FPGA to meet the demands of real-time performance.

REFERENCES

• 1: M. Fikret Ercan and Changjiu Zhou, "Vision based motion planning of humanoid robots," Robotic Welding, Intelligence and Automation, Vol. 299, pp. 212--227, 2004.
• 2: Vladimir Vezhnevets, Vassili Sazonov, and Alla Andreeva, "A survey on pixel-based skin color detection techniques," in Proceedings of the Graphicon 2003, pp 85--92, 2003.
• 3: Yingen Xiong , Bing Fang , Francis Quek, Extraction of Hand Gestures with Adaptive Skin Color Models and Its Applications to Meeting Analysis, Proceedings of the Eighth IEEE International Symposium on Multimedia, p.647-651, December 11-13, 2006 [doi>10.1109/ISM.2006.75]
• 4: C. Torres-Huitzil and M. Arias-Estrada, "Configurable Hardware Architecture for Real-time Window-based Image Processing," Lecture Notes in Computer Science, vol. 2778, pp. 1008--1011, 2003.
• 5: Thomas H. Drayer , Joseph G. Tront , Richard W. Conners , Philip A. Araman, A Development System for Creating Real-time Machine Vision Hardware Using Field Programmable Gate Arrays, Proceedings of the Thirty-Second Annual Hawaii International Conference on System Sciences-Volume 3, p.3046, January 05-08, 1999
• 6: C. Torres-Huitzil, S. E. Maya-Rueda, and M. Arias-Estrada, "A Reconfigurable Vision System for Real-time Applications," in Proceedings of the 2002 IEEE International Conference on Field-Programmable Technology, PP. 286--289, Dec 2002.
• 7: P. C. Arribas, "Real Time Hardware Vision System Applications: Optical Flow and Time to Contact Detector Units," Devices Circuits and Systems, vol. 1, pp. 281--288, Nov 2004.
• 8: H. Shinichi, Z. Masakazu, M. Akihiro, and T. Tatsuhiko, "FPGA-Based Real-time Vision System," Journal of Robotics Mechatronics, vol. 12, no. 4, pp. 401--409, 2004.
• 9: Gooitzen van der Wal , Frederic Brehm , Michael Piacentino , James Marakowitz , Eduardo Gudis , Azhar Sufi , James Montante, An FPGA-Based Verification Framework for Real-Time Vision Systems, Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop, p.124, June 17-22, 2006 [doi>10.1109/CVPRW.2006.27]
• 10: William J. Dally , Steve Lacy, VLSI Architecture: Past, Present, and Future, Proceedings of the 20th Anniversary Conference on Advanced Research in VLSI, p.232, March 21-24, 1999
• 11: Keith Jack, Video Demystified, Elsevier, Oxford, 4th Edition, 2005.
• 12: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.3 SPECIAL-PURPOSE AND APPLICATION-BASED SYSTEMS
      Subjects: Real-time and embedded systems

General Terms:
Algorithms, Design, Performance

Keywords:
fpga, hardware design, skin detection