Identification of Mature Nutmeg using Colour Space Segmentation Algorithm

Introduction

Nutmeg, Myristica fragrans is a perennial aromatic spice that belongs to the Myristicacea family and is native to Indonesia. Globally, India is the second-largest producer⁸ of nutmeg yielding around 16,000 metric tons annually. The states of Kerala, Maharashtra, Tamil Nadu, and Karnataka are the main nutmeg cultivating regions in India.¹⁰

Nutmeg trees can reach a height up to 25 feet, with wide branches. The Nutmeg tree is known for their unique sweet and slightly spicy flavor, and produces two separate spices called nutmeg and mace. The Nutmeg fruits are highly valued for their medicinal values, especially for the issues related with cancer, kidney diseases, Nausea, and intestinal problems.¹⁹ The Nutmeg varieties include KAU- Kochukudy, IISR Keralasree, Kadukanmakal, Kallivayal, Kinattukara, Mundathanam, Konkan Sreemanti, Pullans, Konkan Suganda, Konkan Swad, Naveswari, KAU-Punnathanam, Edavarembil Gold, and Local Clones.IISR Viswashree  developed by Indian Institute of Spices Research (IISR) is the most widely used variety of nutmeg. It yields over 1000 fruits from the eighth year of planting whereas a typical nutmeg tree takes around 20 years to reach its peak harvesting period.

Harvesting the matured nutmeg using the traditional process is a crucial process. Identifying and plucking the matured nutmeg is difficult and more skill is required. Harvesting immature nutmegs without proper knowledge leads to the economic fall, less harvesting rate and increased fatigue for labour. Traditionally, lack of labours is the main challenge faced by the farmers. Existing machine harvesting systems cannot completely replace manual fruit plucking skills. Researchers are trying to develop effective fruit harvesting methods in order to attain the efficiency. To tackle the issues in agricultural sector, numerous automation techniques have been developed using image processing techniques.²² Harvesting using image processing techniques make it challenging because of the environmental factors.

The nutmeg fruit is either plucked from the tree or allowed to drop on the ground and is handpicked. The aim is to reduce human effort by promoting automation concepts in agriculture.¹⁸ Automated identification of fruit with image processing is a keystone in agriculture. The work presented in this paper automatically identifies the mature fruits using image processing techniques. Image segmentation technique is considered, and the methods are explained briefly in the below sections. Image segmentation algorithms like Boundary Edge Detection algorithm, Color Detection Method and Colour Space method were considered.

The design and development of mango fruit harvesting with Shear type method¹⁷ was introduced.50% damage is reduced. A machine vision system was implemented to identify the defects of an apple with accuracy 90%.¹⁶ Previous research mentioned different fruit harvesting sensors using image processing techniques and soft computation methods.² Support vector machine,⁶ artificial neural network algorithm⁴ are the other implementation techniques used for fruit harvesting. The complexity of the system is increased with these methods and more training samples are required to display the exact output. Hyper spectral cameras,¹¹ Thermal imaging cameras,²³ color cameras,  are used for data collection. Generally, the colour of the fruit, the texture and the geometry are the important features which helps to distinguish fruit from the leaves, branches, and other background objects in the farm. Also, clustering, lighting conditions,¹⁴ occlusion¹ are the main challenges faced with the detection of fruits. By applying Shape color based method¹⁵ 85% of accuracy is achieved. Later on, another method was developed in which classification and recognition of the fruits is done by adding colour to the fruits.

Citrus counting algorithm¹³ was developed by converting RGB to HSV colour space. The correlation coefficient, 0.93 was achieved by this method. But, training phase and the testing phase is very difficult. Image processing techniques⁹ for weight prediction and the disease in the fruit was developed for mango¹⁷ grapes and apple. Back propagation method was used and 90% accuracy is achieved.

The objective is to identify matured nutmeg fruits using different methods: boundary edge detection, color detection, and color space conversion. The color space method performs better than the other two image segmentation algorithm, achieving 96% accuracy in identifying matured nutmeg images. Mature nutmegs tend to have specific color characteristics that distinguish them from immature ones. Identifying these differences, making it faster and more efficient compared to manual sorting using a color space segmentation algorithm

Additionally, the average elapsed time for this method is 1.150 seconds. This method also requires less processing time compared to other methods in identifying the right matured fruit. 

Methodology 

Initially the survey is done on various natures of the farms and traditional challenges are identified. There are many different clones available in the nutmeg species, more data on these clones and their varieties has been collected in order to train the image processing system. An image processing algorithm is developed for sensing the right ruptured nutmeg fruits. The ruptured fruit is identified by the selection of the boundaries in the mature fruit. A database is made using images from various nutmeg clones and types. This database trains the algorithm to improve nutmeg maturity detection, which ultimately boosts profits for farmers.

The Collected data sets were trained with image processing algorithms.Three image processing algorithms were incorporated, and the results were carefully analyzed and noted for further modifications. The ruptured fruit is identified by the selection of the boundaries in the mature fruit. Real time data sets are created for few clones and are given below. The figure 1-5 displays different nutmeg varieties, including Kochukudy, Keralashree, Punnathanam, and their corresponding mature and immature fruit images.

Figure 1: Matured Kochukudy Nutmeg                    Click here to view Figure

Figure 2: Matured fruit as a bunch Click here to view Figure

Figure 3: Immature Keralashree Fruit                    Click here to view Figure

Figure 4: Bunch of Punnathanam clone                        Click here to view Figure

Figure 5: Images of Local Clones Click here to view Figure

When the collected data were comprehended, it was observed that most of the fruits were on a solo pattern but there were rare cases where the fruits bear as clusters. Hence, a new challenge evolved to isolate the matured fruit from the cluster and the image processing methodology has to be tuned based on the requirements.

Boundary Edge Detection algorithm, Color Detection Method and Colour Space method are the three image segmentation algorithm used to point out the mature fruit. In Boundary Edge Detection algorithm,²⁴ it was observed that while using this algorithm, immature fruits were not identified in an efficient manner. Therefore Color Detection Method is used. It was observed while employing color detection method, it was noted that the processing time varies for different clones of the nutmeg.Therfore we go for colour space method.

Fig 6 shows the steps involved in the Identification of Mature Nutmeg using segmentation algorithms. The input image is pre-processed with a Guassian filter, which helps to reduce noise and enhance edges. Then the image is converted from RGB color space into HSV color space. To improve the segmentation, we use erosion to remove small noise and dilation to close gaps in the segmented regions. A specified radius of 10 pixels is used to define circular regions around identified contours. Finally, matured nutmeg is identified and harvested.

Figure 6: Identification   of   Mature   Nutmeg   using segmentation Algorithms Click here to view Figure

Colour space Method

In Colour Space method, the images of the different clones of nutmeg were captured using the camera from different regions. The obtained RGB images were processed using the image processing system. The obtained nutmeg images were analyzed using different algorithms and the following results were obtained. The species used for the identification of mature nutmeg are KAU Kochukudy, IISR Keralashree, KAU Punnathanam, and local clone.

In this method the input colour image of the nutmeg is converted to binary image and from the binary image the Region of Interest (RoI) is extracted by using a binary mask⁷. In the Binary mask image, pixels that belong to the ROI are set to 1 and pixels outside the ROI are set to 0. The extracted Mature Nutmeg using Colour space method is displayed below.

Figure 7: Input Image             Click here to view Figure

Figure 8: Binary images Click here to view Figure

Figure 9: Extracted Mature Nutmeg using Colour space method. Click here to view Figure

With image segmentation algorithm, the mature fruit is identified among the immature fruit. Some varieties of nutmeg images with the available data base. Fig 10-14 shows the input image with their respective binary image and the extracted matured image of the species KAU Kouchukudy and IISR Kerala Shree.

Figure 10: Input Image  Click here to view Figure

Figure 11: Binary image Click here to view Figure

Figure 12: Extracted Mature Nutmeg                 Click here to view Figure

Figure 13: Binary image Click here to view Figure

Figure 14: Extracted Mature Nutmeg  Click here to view Figure

Analysis

For the result analysis, the Boundary Edge detection algorithm and colour detection method is compared.

Boundary Edge Detection algorithm

In this method, the mature nutmeg image was captured; the colour image was converted into binary image and then given as an input to the system. The Boundary Edge Detection algorithm was done on the binary image to detect the mature nutmeg fruit with the help of its split boundaries.²² The mature and immature fruits were identified by using this algorithm. However, it may be somewhat time-consuming when identifying mature nutmeg in a large group. Gaussian smoothed step edge (an error function) is used as the simplest extension of the ideal step edge model for modelling the effects of edge blur in practical applications

Thus, a one-dimensional image ‘f’ that has exactly one edge placed at {\displaystyle x=0}x=0 may be modelled as

At the left side of the edge, the intensity is

And right of the edge the intensity i

The scale parameter is called the blur scale of the edge. Ideally this scale parameter should be adjusted based on the quality of image to avoid destroying true edges of the image. The below figure shows the binary image of matured nutmeg, identification of nutmeg  in cluster and immature nutmeg images using boundary detection method. Figures 15 to 17 displays the simplified image of mature nutmeg, the process of identifying nutmeg in clusters, and the identification of immature nutmeg using boundary detection.

Figure 15: Binary image of matured Nutmeg    Click here to view Figure

Figure 16: Identification of Nutmeg in cluster              Click here to view Figure

Figure 17: Identification of Immature nutmeg using Boundary detection method  Click here to view Figure

Colour Detection Method

The color detection method is more effective¹⁹ in identifying clustered fruits compared to the earlier boundary detection method. In this colour detection method the raw image is given to the image processing system. The identification of the mature and immature fruit is indicated using the shape of the fruit. The extracted Mature Nutmeg using Colour Detection method is shown below. This method has been analysed on the different clones and it is observed that the processing time differs from clone to clone. Colour quantization is usually done using the “straight-line distance” or “nearest colour” algorithm, which simply takes each colour in the original image and finds the closest palette entry, where distance is determined by the distance between the two corresponding points in three-dimensional space. In other words, if the colours are (r₁,b₁,g₁)² and (r₂,b₂,g₂)² {\displaystyle (r_{1},g_{1},b_{1})}

Fig 18 shows the input image of the nutmeg. Fig 19 shows the binary image of the nutmeg. The mature nutmeg, obtained through the color detection method, is presented in Figure 20.

Figure 18: Input Image                                    Click here to view Figure

Figure 19: Binary image Click here to view Figure

Figure 20: Extracted Mature Nutmeg using Colour Detection method Click here to view Figure

Results and Discussions

Three image processing algorithms were developed and compared to find the mature fruits in a Nutmeg tree. The results displayed below shows the different varieties of nutmeg collected from the different farms and simulation output for the extracted matured images. By using Boundary Edge Detection and Color Detection Method algorithm, it was noted that, immature fruits were not identified in an efficient manner. Therefore Color space method is used. By using this method, the mature fruits can be identified. The processing time is less and efficiency is high when compared to the previous method. The Nutmeg varieties collected are KAU Kochukudy, IISR Keralashree, KAU Punnathanam and local clone. Fig 21 shows the simulation output for KAU Kochukudy. Processing time for KAU Kochukudy is 1.35 sec. Fig 22 shows the simulation output for IISR Keralashree. Processing time for IISR Keralashree is 1.17 sec. Fig 23 shows the simulation output for KAU Punnathanam. The processing time for KAU Punnathanam is 1.161 sec.

Figure 21: Simulation output for KAU Kochukudy Click here to view Figure

Figure 22: Simulation output for IISR Keralashree               Click here to view Table

Figure 23: Simulation output for KAU Punnathanam Click here to view Figure

Fig 24 shows the simulation output for Local Clones. The processing time for local clone is 1.162 sec

Figure 24: Simulation output for local clone Click here to view Figure

By using Colour Space method, the processing time to identify the mature nutmeg is reduced. The table below shows the comparison for the processing time of different varieties of Nutmeg for the above mentioned three image segmentation algorithms.

Table 1: Comparison of Processing times of different varieties of Nutmeg 

Clones	Edge Detection Method (sec)	Colour Detection Method (sec)	Coloure Space Method (sec)
KAU Kouchukudy	1.56	1.35	1.2
IISR KeralaShree	1.38	1.17	1.05
Punnathanam	1.80	1.161	1.042
Kodumamackal	1.32	1.173	1.05
Local Clone	1.164	1.162	1.044

The table above displays the processing times for various Nutmeg varieties, comparing both the current algorithm and the new approach. The figure illustrates that the proposed method significantly reduces processing time.

Figure 25: An illustration of the processing times for various nutmeg species in a bar chart      Click here to view Figure

Conclusion

We have come up with the solution for detection of matured nutmeg spice. Image segmentation algorithm was developed to harvest nutmeg spices. Three new algorithms were developed. Boundary Detection, Colour Detection and Colour Space methods are used for computation. Fruit identification using image processing is a difficult task. We attain better results when compared to the previous work. The Simulation results of Boundary detection, Colour detection and colour space methods are compared. Colour space method outperforms the other two methods in terms of identification of matured nutmeg images with 96% accuracy. The proposed methods can easily identity the matured nutmeg fruits based on color and texture features. As the nutmeg is identified using the Colour space method, the processing time to identify the right matured fruit requires much lesser time than other methods. The average elapsed time was 1.150 secs was achieved. Fast recognition of the mature fruits is the main advantages of this system. For Future work, it is planned to develop an automatic handy and portable nutmeg harvesting machine without damaging the mace for nutmeg harvesting. 

Acknowledgement

The authors are grateful to acknowledge the financial support of Department of Science and Technology under TDP scheme, [DST/TDT/AGRO-38/2022] New Delhi for carrying out this Research project. The authors are expressing their sincere thanks to the Management of Mar Ephraem Engineering College, Elavuvilai for their constant encouragement and support.

Conflict of Interest

There is no conflict of interest.

Funding Sources

There are no funding sources.

Reference

1. Brosnan, Tadhg, and Da-Wen Sun. Improving quality inspection of food products by computer vision––a review. Journal of food engineering61, no. 1 (2004): 3-16. M. Bulanon, T. F. Burks and V. Alchanatis, Study of temporal variation in citrus canopy using thermal imaging for citrus fruit detection, Biosyst. Eng. 101 (2008): 161–171.
   CrossRef
2. Bulanon, D. M., T. F. Burks, and V. Alchanatis. Improving fruit detection for robotic fruit harvesting. In International Symposium on Application of Precision Agriculture for Fruits and Vegetables 824, pp. 329-336. 2008.
   CrossRef
3. Wang, Yutan, et al. Image segmentation and maturity recognition algorithm based on color features of Lingwu long jujube. Advance Journal of Food Science and Technology12 (2013): 1625-1631.
   CrossRef
4. Choudhary, Pradeepkumar, Rahul Khandekar, Aakash Borkar, and Punit Chotaliya. “Image processing algorithm for fruit identification.  Res. J. Eng. Technol.(IRJET)4, no. 3 (2017).
5. Bennedsen, B. S, D. L. Peterson, and Amy Tabb. Identifying defects in images of rotating apples.Computers and Electronics in Agriculture no. 2 (2005): 92-102.
   CrossRef
6. Lopez, Jose J., Maximo Cobos, and Emanuel Aguilera.Computer-based detection and classification of flaws in citrus fruits. Neural Computing and Applications20 (2011): 975-981.
   CrossRef
7. Pachangane, A. A., K. G. Dhande, and V. S. Nalawade. Performance Evaluation of DBSKKV Developed Fruit harvesters for Matured Nutmeg Harvesting. International Journal of Environment and Climate Change7 (2021): 125-132.
   CrossRef
8. Lino, Antonio Carlos Loureiro, Juliana Sanches, and Inacio Maria Dal Fabbro. Image processing techniques for lemons and tomatoes classification.Bragantia67 (2008): 785-789.
   CrossRef
9. Hung, Calvin, James Underwood, Juan Nieto, and Salah Sukkarieh. A feature learning based approach for automated fruit yield estimation. In Field and Service Robotics: Results of the 9th International Conference, pp. 485-498. Springer International Publishing, 2015.
   CrossRef
10. Dubey, Shiv Ram, and Anand Singh Jalal. Species and variety detection of fruits and vegetables from images.International Journal of Applied Pattern Recognition1, no. 1 (2013): 108-126.
    CrossRef
11. Sapowadia, B. D., H. N. Patel, R. A. Gupta, and S. R. Pund. Design and development of a mango harvesting device. Agricultural mechanization in asia africa and latin americano. 1 (2001): 31-34.
12. Dharmasiri, S. B. D. H., and S. Jayalal. Passion fruit disease detection using image processing.In 2019 International Research Conference on Smart Computing and Systems Engineering (SCSE), pp. 126-133. IEEE, 2019.
    CrossRef
13. Hayashi, K. Ganno, Y. Ishii, and I. Tanaka. Robotic harvesting system for eggplants. Japan Agricultural Research Quarterly. 36:163–168, 2002.
    CrossRef
14. Liu, Y., B. Chen, and J. Qiao.Development of a machine vision algorithm for recognition of peach fruit in a natural scene. Transactions of the ASABE54, no. 2 (2011): 695-702.
    CrossRef
15. Saini, Sujata, and Komal Arora.A study analysis on the different image segmentation techniques.International Journal of Information & Computation Technology14 (2014): 1445-1452.
16. Hamam, atef s., mahmoud e. El-iraqi, youssef f. Sharibim, and taher r. Awais. design and evaluation of mechanical picking heads for citrus fruits harvesting. egyptian journal of agricultural research89, no. 4 (2011): 1545-1562.
    CrossRef
17. Sonam Saluja1, Aradhana Kumari Singh2, Sonu Agrawal.A Study of Edge-Detection Methods.International Journal of Advanced Research in Computer and Communication Engineering .Vol. 2, Issue 1, January 2013
18. Arefi, Arman, Asad Modarres Motlagh, Kaveh Mollazade, and Rahman Farrokhi Teimourlou.Recognition and localization of ripen tomato based on machine vision.Australian Journal of Crop Science no. 10 (2011): 1144-1149.
19. Arivazhagan, Shebiah, R. Newlin Shebiah, S. Selva Nidhyanandhan, and L. Ganesan. Fruit recognition using color and texture features.Journal of Emerging Trends in Computing and Information Sciences no. 2 (2010): 90-94.
20. S. Bennedsen and D. L. Peterson, Performance of a system for apple surface defect identification in near-infrared images, Biosyst. Eng. 90 (2005): 419–431.
    CrossRef
21. Whittaker, Dale, et al. Fruit location in a partially occluded image.Transactions of the ASAE3 (1987): 591-0596.
    CrossRef
22. Pachangane, A. A., K. G. Dhande, and V. S. Nalawade. Performance Evaluation of DBSKKV Developed Fruit harvesters for Matured Nutmeg Harvesting.International Journal of Environment and Climate Change11, no. 7 (2021): 125-132.
    CrossRef
23. Gurusamy, Vairaprakash, Subbu Kannan, and G. Nalini. Review on image segmentation techniques. J Pharm Res20125 (2013): 4548-4553.