This research paper investigates the applicability of existing Convolutional Neural Network (CNN) models to cultural image data, with a specific focus on Indian Petroglyphs. CNNs, renowned for their success in image classification tasks, are typically trained on diverse datasets. However, their strengths in handling generic visual information may not extend to the complex characteristics of cultural artifacts. This chapter explores the mechanisms of CNN models, particularly VGG16, in identifying images with the aid of Layer-wise Relevance Propagation (LRP) that is an Explainable Artificial Intelligence technique (XAI). While CNNs exhibit remarkable accuracy in generic image classification, their limitations become apparent when applied to cultural data, such as Indian Petroglyphs, particularly those from Saru-Maru Caves, Madhya Pradesh, India. By shedding light on the limitations of existing deep learning models, this research contributes valuable insights for refining approaches to cultural image data analysis and the need for domain-specific experts to generate training datasets with special focus on the specific intricacies posed by the petroglyphs under study.