In this paper, we introduce a novel algorithm to destroy the steganographic information embedded in an image without changing the quality of the image and with no prior knowledge of the used steganography scheme. We propose the new Neighbor Class Displacement (NCD) algorithm, which arranges the pixels of an image into a given number of classes. The elements of two specific classes are substituted with each other based on different conditions related to the content of the class elements. For evaluating the effectiveness of our attack, we apply NCD to different steganographically modified images to remove the embedded hidden information. Our results show that over 40% of the steganography bits are toggled in natural images by our proposed attack algorithm, which means the hidden information is removed effectively. Additionally, the visual quality of the images does not change and the PSNR of the original and attacked images is above 32 dB. We compare our attack to other signal processing and geometrical attacks and show that our NCD scheme outperforms other steganography attack algorithms while maintaining the quality of the host image.