COMPUTER VISION SYSTEM TO ESTIMATE CASHEW KERNEL (WHITE WHOLES) GRADE GEOMETRIC AND COLOUR PARAMETERS

V.G. Narendra¹, K.S. Hareesh^2
1 Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal University, India
² Department of Computer Applications, Manipal Institute of Technology, Manipal University, India

ABSTRACT

The geometric parameters along with related colour properties of food and agricultural products are important in order to characterize and describe its quality. Presently, the quality assessment of food and agricultural products on the basis of visual inspection are done by human, so it is time consuming and laborious. The alternative of visual inspection is computer vision system, with the application of image processing technique for this purpose can certainly reduce the human drudgery while guaranteeing the quality of food and agricultural products. In this paper, a new colour and geometric parameters estimation methods suggested, to provide automatic and intuitive way of quality inspection of cashew kernel from an image accurately. The geometric parameters are obtained manually and proposed method, and we have estimated the average values (i.e. minimum and maximum) of length, width and an area of cashew kernels (white wholes) grade. Finally, compared the results of geometric parameters using proposed method and manually done. To identify the representation of colour values in suitable colour space for cashew kernel, we have investigated the hardware-oriented (i.e. RGB color space), human-oriented (i.e. HSV color space) and instrumental (i.e. CIEL*A*B* color space). It is estimated from our research that, the statistical measurements of cashew kernels (white wholes) grade colour parameters from the proposed methods.  Finally the paper concludes by highlighting the necessity of such computer vision system and insight into the methodology that attempts to fix the universal estimation of colour and geometric  parameters measurement of cashew kernels (white wholes) grade.

Key words: Quality, Geometric parameters, Colour parameters, Cashew Kernels, Colour Space Computer Vision.

1. INTRODUCTION

Cashew (Anarcardium Occidentale L) is native of Brazil. Cashew were brought to India by the Portuguese travelers during the 16^th century for afforestation and soil conservation purpose. Cashew has emerged as an important plantation crop of India and it plays a significant role in Indian economy [3]. Cashew earned foreign exchange equivalent to 43,900 million rupees, from export of 1,32,000 MT of cashew kernels during the year 2011–2012 [22]. India is one of the three major largest processors, exporters and the second largest consumer of cashew kernels in the world, after the USA [23]. Cashew kernels are obtained from raw cashew nuts, the true seeds of the cashew tree. Cashew are most delicious considered to be among many other edible tree nuts. It is proved that cashew add taste virtually to anything i.e. Ice Creams, Sweets, Chocolates, and various Dishes [1].

In India cashew farms and processing units are located in states of Maharastra, Goa, Andhra Pradesh, Orissa, West Bengal, Karnataka, Kerala, Tamilnadu, Madhya Pradesh, and in the eastern regions. In factory, composite lots of raw cashew nuts, after processing, yield the bulk of cashew kernels of varied size and weights, which in turn decide the marketability of the kernels under different grades. Grading of cashew kernels is a prerequisite to meet the requirements of domestic as well as international trades and marketing [1]. Cashew kernels are graded depending on their size, shape and colour standards specified by the Export Control and Inspection Act, 1963 [5]. There are as many as 26 cashew kernel grades available in the market ranging from wholes to pieces of which W-180, W-210, W-240 and W-320 are the important grades in the global market. The important characteristics of the white wholes cashew kernels are as mentioned in the Table 1 [5]. The current standards for cashew kernels (white wholes) grade are ambiguous, because of count per 454 grams size description and general characteristic as mentioned in the Table 1.

A major problem in sorting or grading of cashew kernels is the use of harmful mechanical equipment or expensive colour sorters; still, cashew kernels grading is carried out manually. Manual sorting or grading is based on traditional visual quality inspection performed by trained labours. The problem inherent in this system includes high labor costs, worker fatigue, inconsistency, variability, and scarcity of trained labor. Today, various kinds of cashew kernel grades are provided in the market with different qualities, due to the variability in quality decisions among the graders. Due to the scarcity of trained labor, mechanization at various stages of cashew processing is gaining importance in this country and abroad. This way of grading presents many quality problems and grading is the final opportunity for the quality control [15]. To ascertain the quality [4], the cashew kernel (white wholes) grade standards have been developed by considering the geometric and colour parameters.

Computer-based vision technology offers a high level of flexibility and repeatability at relatively low-cost with fairly high throughput with superior accuracy. Computer vision offers an alternative to visual physical inspection, and has been in numerous foods and agricultural commodity sorting or grading systems today. Moreover, this practice is  objective, consistent, rapid and economical. Colour and size are the primary information sources for foods and agricultural commodity (i.e. object) inspection, classification, and sorting or grading [7]. Computer vision systems have been effectively used to classify or to recognize quality parameters like colour and size in several agricultural and food commodities including dry beans [14], pistachios [11], coffee [6], soya beans seeds [21], peanuts [12], brazil-nuts [24, 28], pizza [26, 27] and potato chips [20] advancement in hardware and image processing makes computer vision a very popular technique for automatic cashew kernel quality inspection of parameters like colour and size. 

At present, there are no automated systems for cashew kernel (white wholes) grade, to estimate standard quality parameters like colour and geometric. Our primary objective of this study is to develop a cost-effective intelligent computer vision system to estimate the standard cashew kernel (white wholes) grade colour and geometric parameters from an image samples.

2. MATERIALS AND METHODS

2.1. Samples Collection
The samples of cashew kernel (white wholes) grade (i.e W-180, W-210, W-240 and W-320 of 750 samples each) used in this study were collected from Achal Industries Mangalore, Balaji Cashew Exports Jarkal, Mahalasa Cashew Exports Hiriyadka, Tirumala Cashew Industries Karkal and Sathyshree Cashew Industries Hebri located in coastal Karnataka, India. 

2.2. Estimation of Geometric parameters (i.e. Size) of Cashew kernel
Because of irregular shape of food objects, the calculation of the length and width is much more complex than that of area and perimeter [16]. Nevertheless, some measurements for length and width have been developed by researchers and are used in the food industry. The measurements most commonly used are Feret’s diameter, by calculating the major axis and the minor axis [31] of an object.

2.2.1. Feret’s Diameter
Feret's Diameter is also called the caliper length. It is the longest distance between any two points in the boundary of the region of interest [24, 28] as shown in Figure 1.

All the cashew kernels of different grades are looking like an elliptical shape[16]. So we considered the major and minor axis as an actual length (L) and width (W) of cashew kernel [2] as shown in Figure 2.

2.2.2. Measurement of Cashew kernel using Instrument (i.e. Vernier Caliper)
Vernier caliper instrument of least count 0.02 mm was used to measure the actual length and width of the cashew kernel of cashew kernel (white wholes) grade as shown in Figures 3a and 3b [17]. The average values (i.e. minimum and maximum) of each cashew kernel (white wholes) grade length (L), width (W) and calculated an area (i.e. A  =  Π*L*W) are tabulated in Table 2. The measurement of a single cashew kernel length and width are 6 to 7 Seconds by using Instrument Vernier Caliper.

2.3. Computer Vision System
A computer vision system which is developed consists of two web cameras in front and top of Cashew kernel sample under investigation at a distance of 10 cm from the sample position and perpendicular to each other, a colour matching cabinet, Artificial Daylight Fluorescent Lamps D-65 (18 W and 6500 K), and a computer System as shown in Figure 4.

2.3.1. Image acquisition
Image acquisition involves capturing an image of resolution 640 × 480 (i.e. cashew kernel) added to database (raw file format i.e. j2c) by using two webcams (iBall and SmartPC, each of 12 Mpixels still image resolution) which are less expensive compared to camera, in a colour matching cabinet with a proper control of lighting intensity under Artificial Daylight Fluorescent Lamps (D-65). Distances and heights between webcams and cashew kernel were set for all the samples with moisture content in the range of 3.5–5%.

2.3.2. Pre-processing and Segmentation
Image processing Toolbox in the MATLAB 2012a is invoked as an image analysis and a processing software to extract the features from the cashew kernel image. Figure 4 shows the general operations for the cashew kernels (white wholes) grading system. During preprocessing phase, image is smoothened using 5 × 5 median filter mask. In this study, black colour background is used to have a bimodal histogram. Threshold Segmentation Technique differentiates the cashew kernel region into the binary image.

2.3.3. Feature Extraction
After getting the boundary of the selected region, properties such as area, major and minor axis length can be determined. The area is the actual number of pixels in the labelled region. The major and minor axis length can be the actual length and width of a cashew kernel, after multiplying by pixel dimensions. This measurement identifies the geometric parameters of a cashew kernel.

2.3.3.1. Geometric parameters (Size)
We have proposed an algorithm to estimate the geometric parameters of a cashew kernel (white wholes) grade from an image samples by calculating pixels per inch [9].

Algorithm 01: Estimate the geometric parameters of a cashew kernels (white wholes) grade

Start

Step 1: Read the image of resolution 640x480 (i.e. cashew kernel)

Step 2: Calculation of a monitor pixels per inch (PPI): 

i) Calculate diagonal resolution in pixels using the pythagorean theorem:

(1)

ii) Calculate PPI:

(2)

where:
d_p is diagonal resolution in pixels,
w_p is width resolution in pixels,
h_p is height resolution in pixels and di is diagonal size in inches.

Step 3: Smooth the image using median filter (remove the noise)

Step 4: Binarize the image using OSTU threshold technique

Step 5: Calculate A  =  counted white pixels,

The area A_i is measured in pixels and indicates the relative size of the object.

where:
I_i (r, c)  =  1 if I(r,c)  = i^th object 0 otherwise.

We used 1 inch  =  2.54 cm to convert Area to centimeter in equation 4.

Step 6: Find the angle θ to rotate the image:

An angle θ to rotate the cashew kernel image is obtained using second order central moments as in equation (5). The angle is calculated  using the equations from (6) to (13), where f (x, y) represents the grey level in the point (x, y) , the point (x, y) is the centroid of the cashew kernel image, U(2,0) , U(0,2) and U(1,1) second order central moments, M(1,0) and M(0,1) first order moments and M(0,0) the zero order moment.

θ  =  1/2*tan-1(2*U(1, 1) / (U(2, 0) – U(0, 2)))

(5)

where:

(6)

(7)

(8)

(9)

(10)

(11)

=  M(1,0) / M(0,0)

(12)

=  M(0,1) / M(0,0)

(13)

Step 7: After obtaining θ, it is computed for each pixel (v, w) in the binarized image, the new spatial location (x, y) in the aligned cashew kernel image through equations (14) and (15).

Step 8: The new centroid (, ) or center of mass in the alignment cashew kernel image, which is the geometric center of the object’ shape (i.e. cashew kernel), is calculated again using equations (12) and (13).

Step 9: The size of the longest horizontal erosion is Horizontal_Pixel_Count. This defines the  quantity of points of the cashew kernel width obtained through successive directional erosions of a minor axis with a slope of 0°. The cashew kernel width measured in centimeters is set as the ratio of 

The size of the longest vertical erosion Vertical_Pixel_Count is calculated.
This variable defines the number of points of the cashew kernel length.
This is obtained through successive directional erosions of a major axis with a slope of 90°. The cashew kernel length is also measured in centimeters and is set as the ratio of Vertical_Pixel_Count and PPI as in equation (18).

The actual Length and Width can be obtained by multiplying with 2.54 cm because 1 inch  =  2.54 used in equation 17 and 18.

Stop

The algorithm 1 is used to obtain length and width of white wholes grade cashew kernel from image takes processing time as 0.092 Seconds.

2.3.3.2. Colour Spaces
Colour provides the basic information for human perception. Colour is also elementary information that is stored in pixels to constitute a digital image. Hence colour is rated as one of the most prominent features for object measurements, image understanding and object description [24, 28].

Three typical statistical measurements, including the mean, variance and standard deviation are noted in each component as colour measurements. Different types of values stored for the three colour components, and different colour reproduction methods using these three values (i.e. Red, Green and Blue) lead to different colour spaces [30]. These spaces can be generally classified into three categories: hardware-oriented, human-oriented and instrumental. Measurements of colour are dependent on these spaces [24, 28].

To explore variations in different colour spaces, we have experienced with following colour spaces are as follows.

2.3.3.2.1. RGB colour space
The most popular hardware-oriented space is the RGB (red, green, blue) space. RGB is therefore used in most computers, for image acquisition, storage, and display [27].

Colour images are supplied by a web camera and saved in the three-dimensional RGB (red, green, and blue) colour space with raw format (i.e. j2c file format). RGB colour image is a M × N × 3 array of colour space pixels, where each colour pixel is a triplet corresponding to the red, green, and blue components of a RGB image at a specific spatial location [10].

2.3.3.2.2. Hue Saturation and Value (HSV) colour space
Hue is measured by the distance of the current colour position of red axis, which is manifested by the difference in colour wavelengths. Saturation is a measurement of the amount of colour – i.e. the amount of white light that is present in the monochromatic light [13]. The final component – intensity, value, or lightness – refers to the brightness or luminance, defined as the radiant intensity per unit projected-area by the spectral sensitivity associated with the brightness sensation of human vision [11].

The relation between the RGB space and the HSV space can be described by the following equations.

(19)

(20)

(21)

2.3.3.2.3. CIE XYZ colour space
The CIE XYZ colour space takes the human eye as a basis, it forms the base for all colour management systems and includes all perceivable colour. This colour space is determined by CIE in 1931 according to standard illuminant (D65) and standard observer (2°) [24, 28].

The following equations can be used to convert colour measurements linearly from RGB space to XYZ space [24, 28]. Firstly, nonlinear R, G, and B values are converted to linear and nominal values as follows.

If R’,G’,B’ ≤ 0.04045 (IEC 61966-2-1 std,1999)

else

(25)

Converting RGB to CIE XYZ is performed with the help of a 3 × 3 matrix.

(26)

The conversion from Reference white RGB colour space to average artificial daylight fluorescence lamps (D65), CIEXYZ D65 colour space is given below [25];

(27)

2.3.3.2.4. CIE L*a*b* Colour Space
The components are "L" (light), the rate from green to red "a", from blue to yellow "b". Discovering the colour components of CIE L*a*b* colour space is possible with CIE XYZ colour space. The Observer and the standard illuminator used to discover X, Y, and Z values affect these values directly. Transformation from CIE XYZ to CIE L*a*b* can be obtained by the following equations [16].

(28)

(29)

(30)

(31)

 X_n, Y_n, and Z_n are the tristimulus values of the illuminant, in this case illuminant is D65.  The colour of the Cashew kernel is determined according to the intensity and dispersion of the colour in the specified area. From CIE L*a*b* colour space, In this study, we have calculated the chroma, colour score (CS) and white index (WI) of Cashew kernel using following equations [29]. From chroma, colour score and white index, we can easily differentiate the cashew kernels (white wholes) grade.

Finally we calculated mean, variance, standard deviation and range using following equations.

(35)

(36)

(37)

(38)

2.3.3.3. Colour Measurement
Twelve features of RGB colour (mean, variance, standard deviation and range of red, green and blue channels) are extracted from the cashew kernel image. Similarly, other twelve features of HSV colour (mean, variance, standard deviation and range of hue, saturation and value) are extracted from the image of cashew kernel.

We have estimated the colour parameters from RGB and HSV color space i.e. converting RGB to HSV by using equations 19 through 22, takes processing time as 0.0256 Seconds for a single cashew kernel image. We were calculated Mean, Variance, Standard Deviation and range of RGB as well as HSV components using equations 35 to 38. The average mean and standard deviation values are tabulated in Table 4 and 5 of white wholes grade cashew kernel.

Similarly, the RGB colour space is again converted to CIE L*a*b* colour space using equations (23) through (31) and then the corresponding twelve colour features (i.e. Twelve) are extracted using proposed method i.e. algorithm 2. The average values of mean and standard deviation of L*, a* and b* were tabulated in Table 6. The results obtained from algorithm 3 are tabulated Table 6 and Table 7.

 Algorithm 02: Estimate the colour parameters from RGB  and CIE L*a*b*Colour space

Input: Original 24-bit colour image of resolution 640x480 (i.e. cashew kernel).

Output: 12 colour features.

Start

Step 1: Separate the RGB components from the original 24-bit input colour image.

Step 2: Obtain the XYZ components from RGB components using the equations (23) through (27).

Step 3: Obtain the L*a*b* components from CIE XYZ components using the equations (28) through (31).

Step 4: Compute Mean, Standard Deviation and Range from L*a*b* components using the equations (35) through (38).

Step 5: Compute chroma, colour score and whiteness index components using the equations (32) through (35) for CIE L*a*b* colour space.

Stop

3. RESULTS AND DISCUSSION

3.1. Geometric Parameters measurement
In this study, initially we have measured all the samples of each cashew kernels (white wholes) grade length as well as width manually using Vernier caliper instrument and calculated an area. The results are summarized in Table 2.

Table 2 clearly indicates the distinct difference between different cashew kernels (white wholes) grade in terms of length, width and area measurements. But the manual measurement are time consuming, less efficient and laborious.

Figures 5a and 5b show the images that resulted from the dilations of size Horizontal_Pixel_Countand Vertical_Pixel_Count, respectively using the proposed method (i.e. algorithm 1).

We have estimated the cashew kernel (white wholes) grade length, width and area using proposed method (i.e. algorithm 1) and an average (i.e. minimum and maximum) values are presented in Table 3.

The Table 3 clearly indicates the distinct differences between different cashew kernels (white wholes) grade in terms of length, width and area measurements. It is clear from the tables (i.e. Tables 2 and 3), there is an absolute difference in length, width and area of each category of cashew kernel (white wholes) grade. And also illustrated in the bar graph as showed in Figure 6. There is a clear indication from the measurements, the proposed algorithm outperforms compared to manually done using Vernier caliper instrument with an acceptable error (i.e. less than 5%), and illustrated in scatter plots shown in Figure 7. From the scatter plots, we can observe the measured length and width values are overlapping each other. Therefore, it is clearly visible from the scatter plot, the proposed metod i.e. algorithm 1 is better compared to manually done.

3.2. Colour Parameters measurement
We have investigated three colour spaces RGB, HSV and CIE L*a*b*. Some of the typical statistical features were measured includes mean, standard deviation and range from RGB (red, green, and blue channels) and HSV (hue, saturation, and value channels) colour components using proposed method i.e. algorithm 2, obtained results are tabulated in Tables 4 and 5.

The corresponding graphical representation is shown in Figure 8a. From Figure 8a, it is clearly visible, all the cashew kernel (white wholes) grade (i.e. W-180, W-210, W-240 and W-320) have a clear and significant distinction from each other in terms of mean, standard deviation and range of statistical measurements of red, green and blue channels. We can observe, the blue channel gives difference in mean and standard deviation values among all cashew kernel (white wholes) grade. Unfortunately, RGB colour space is not consistently uniform, and the proximity of colour does not give colour similarity [8].

HSV space has been developed by considering the concept of visual perception in human eyes; colour measurements obtained from HSV are better related to the visual significance of food (i.e. cashew kernel) surfaces. This is due to greater correlation between the colour measurements from human-oriented spaces and the sensory scores of cashew kernels image. Using HSV colour channels, the statistical measurements were estimated from proposed methods and tabulated in Table 5.

We have observe that, Table 5 clearly indicates the distinctions between different cashew kernels (white wholes) grade in terms of statistical measurements (i.e. mean and standard deviation) of Hue, Saturation and Value channels. The statistical measurements were analyzed and results plotted in Figure 8b, from the figure it is evident that each and every grade has a clear and distinct separation in terms of mean and standard deviation of hue channel.

However, we have also explored CIEL*a*b* colour space, which happens to be one of the widely accepted colour spaces to represent multiple variations, what generally human being perceived. Comparable statistical measurements were calculated using algorithm 3 and the obtained results were tabulated in Tables 6 and 7.

We can observe that Table 6, clearly indicate that, there is a distinct difference between statistical measurements of CIE L*a*b* colour space of cashew kernel (white wholes) grade.

We can observe that Table 7, clearly indicate that, there is a distinct difference between measurements of Chroma, Colour Score and Whiteness index of cashew kernels (white wholes) grade. The consequences were also represented using graphical representation in Figure 9a. The negative colour score gives distinctly difference among cashew kernels (white wholes) grade is shown in Figure 9b. From the obtained results, it is very clear that W-180 and W-210 are very much distinctive, where slight ambiguity in discriminating W-240 and W-320. This ambiguity can be cleared by analyzing range values. The summary of colour and geometric parameters estimation were tabulated in Table 8.

4. CONCLUSIONS

Some important observation drawn from the study of the statistical measurement of cashew kernels (i.e.W-180, W-210, W-240 and W-320) grade are as follows.

1. The geometric parameters of each cashew kernels (white wholes) grade length, width and area are estimated using proposed method (i.e. algorithm 1), and average values (i.e. minimum and maximum) gives distinctly different result from an image samples.
2. The colour parameters are estimated with RGB, HSV and CIEL*A*B* colour space using proposed methods. The average mean and standard deviation of each cashew kernels (white wholes) grade gives distinctly different result from image samples. Among the three color space, we have identified the CIE L*A*B* color space to represent colour values of cashew kernel, because it includes all human perceivable colour according to standard illuminant and observer.
3. From this study, measurement of cashew kernels (white wholes) grade is practically realized considering the geometric and colour values for sorting and grading of cashew kernels.

We conclude that, the estimation of cashew kernels (white wholes) grade colour and geometric parameters from an image samples using proposed methods are less time consuming and more efficient. Finally we obtained the standard estimation of geometric (i.e. length, width and area) and colour parameters (i.e. mean and standard deviation) for each cashew kernels (white wholes) grade. The future work of this research is to build the intelligent computer vision system for sorting and grading of cashew kernels (white wholes) grade by using soft computing technique (i.e. Artificial Neural Network).

Accepted for print: 20.10.2014

Responses to this article, comments are invited and should be submitted within three months of the publication of the article. If accepted for publication, they will be published in the chapter headed 'Discussions' and hyperlinked to the article.