Colonoscopy is currently the gold standard procedure for colorectal cancer screening. However, the dominant explanations for the continued incidence of CRC are endoscopist-related factors. To address this, we have been investigating an automated feedback system which measures quality of colonoscopy automatically to assist the endoscopist to improve the quality of the actual procedure being performed. One of the fundamental steps for the automated quality feedback system is to distinguish a colonoscopy from an upper endos-copy since upper endoscopy and colonoscopy procedures are performed in the same room at different times, and it is necessary to distinguish the type of a procedure prior to execution of any quality measurement method to evaluate the procedure. In upper endoscopy, a bite-block is inserted for patient protection. By detecting this bite-block appearance, we can distinguish colonoscopy from upper endoscopy. However, there are various colors (i.e, blue, green, white, etc.) of bite-blocks. As a result, to detect it irrespective of its colors in a video is a challenge. Though convolution neural network (CNN) has been used for object detection, it can mostly detect an object with very similar col-ors or shapes. In the proposed method, we implement a CNN using grayscale images, so we can detect a bite-block regardless of its colors. The experimental results show that the proposed method is highly promising.