Is it Created by AI?: An Experimental Investigation of Viewers’ Differentiation and Perceived Effectiveness of AI in Promoting a Non-profit Cause
DOI:
https://doi.org/10.35313/jmi.v5i2.356Keywords:
Artificial Intelligence, Media Effects, Experimental Design, AI Content Creation, Digital MarketingAbstract
The expeditious development of Generative Artificial Intelligence (AI) has allowed for an easier application of this technology in crafting communication campaigns. To investigate whether AI is helping or hindering the effectiveness of the communication process to persuade the public to act on a non-profit cause, a comparative experimental study is designed. Through a 2 (AI-generated vs. human-generated) x 3 (text only vs. text and image congruent vs. text and image incongruent) experimental research, we examined participants’ ability to differentiate between AI-generated content vs. human-generated content and their effects. Results from N = 700 participants sampled through Prolific.co indicated that participants could not differentiate between AI-generated and human-generated promotional text; however, they were able to differentiate between the images. Regarding effects, participants perceive human-generated content as more effective, authentic, trustworthy, and appealing through negative emotions, such as worry and anger. Although the study applied the Media Effects research framework through an experimental design, the findings are limited by the non-representative data collection through Prolific.co. The study provides empirical support for the application of generative AI in digital communication, internet marketing and advertising, and higher education in fields of promotional content generation. The study offers experimental support from the audience perspective to the conversation on the feasibility of AI applications in digital content generation in the fields of non-profit marketing and advertising.
Downloads
References
Aktay, S. (2022). The usability of images generated by artificial intelligence (AI) in Education. International Technology and Education Journal, 6(2), 51-62.
Alves, H., Fernandes, C., & Raposo, M. (2016). Social media marketing: a literature review and Implications. Psychology & Marketing, 33(12), 1029-1038. https://doi.org/10.1002/mar.20936
Argyris, Y. A., Wang, J., Kim, Y., & Yin, Z. (2020). The effects of visual congruence on increasing consumers’ brand engagement: an empirical investigation of influencer marketing on instagram using deep-learning algorithms for automatic image classifications. Computers in Human Behavior, 112, 1-15. https://doi.org/10.1016/j.chb.2020.106443
Barton, K. (2020). The do’s and don’ts of using emotions in your communications. Forbes. https://www.forbes.com/sites/forbescommunicationscouncil/2020/10/14/the-dosanddonts-of-usingemotionsinyourcommunications/sh=7bbc81fe37ca#:~:text=Emotions%20have%20power%20in%20communications,the%20keys%20to%20audience%20engagement.&text=Use%20it3A%20If%20you%27ve,people%20know%20you%27re%20angry
Braga, A., & Logan, R. K. (2017). The emperor of strong AI has no clothes: limits to artificial intelligence. Information, 8(4), 1-21. https://doi.org/10.3390/info8040156
Bui, H. T., Filimonau, V., & Sezerel, H. (2024). AI-thenticity: Exploring the effect of perceived authenticity of AI-generated visual content on tourist patronage intentions. Journal of Destination Marketing & Management, 34, 100956. https://doi.org/10.1016/j.jdmm.2024.100956
Capatina, A., Kachour, M., Lichy, J., Micu, A., Micu, A., & Codignola, F. (2020). Matching the future capabilities of an artificial intelligence-based software for social media marketing with potential users’ expectations. Technological Forecasting & Social Change, 151. https://doi.org/10.1016/j.techfore.2019.119794
Cao, J., Li, X., & Zhang, L. (2025). Is relevancy everything? A deep-learning approach to understand the effect of image-text congruence. Management Science, 1-24. https://doi.org/10.1287/mnsc.2022.01896
Cinelli, M. D., & LeBoeuf, R. A. (2020). Keeping it real: How perceived brand authenticity affects product perceptions. Journal of Consumer Psychology, 40-59. https://doi.org/10.1002/jcpy.1123
Corbitt, B. J., Thanasankit, T., & Yi, H. (2003). Trust and e-commerce: a study of consumer perceptions. Electronic Commerce Research and Applications, 203-215. https://doi.org/10.1016/S1567-4223(03)00024-3
Cooke, D., Edwards, A., Barkoff, S., & Kelly, K. (2024). As good as a coin toss: Human detection of AI-generated images, video, audio, and audiovisual stimuli. Arxiv, 1-19.
Dai, Y., Lee, J., & Kim, J.W. (2023). AI vs. human voices: How delivery source and narrative format influence the effectiveness of persuasion messages. International Journal of Human Computer Interaction, 1-15. https://doi.org/10.1080/10447318.2023.2288734
Dillard, J. P., Weber, K. M., & Vail, R. G. (2007). The relationship between the perceived and actual effectiveness of persuasive messages: a meta-analysis with implications for formative campaign research. Journal of Communication, 57(4), 613-631. https://doi.org/10.1111/j.1460-2466.007.00360.x
Duffett, G. D. (2017). Influence of social media marketing communications on young consumers’ attitudes. Young Consumers, 18(1), 19-39. https://doi.org/10.1108/YC-07-2016-00622
Durán, J. M., & Pozzi, G. (2025). Trust and trustworthiness in AI. Philosophy & Technology, 38(16), 1-31. https://doi.org.10.1007/s13347-025-00843-2
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R.,
Edwards, J., Eirug, A., Galanos, V., Ilavarasan, P. V., Janssen, M., Jones, P., Kar, A. K., Kizgin, H., Kronemann, B., Lal, B., Lucini, B., …Williams, M. D. (2021). Artificial intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 57. https://doi.org/10.1016/j.ijinfomgt.2019.08.002
Galloway, C., & Swiatek, L. (2018). Public relations and artificial intelligence: it’s not (just) about robots. Public Relations Review, 44(5), 734-740. https://doi.org/10.1016/j.pubrev.2018.10.008
Göring, S., Rao, R., Merten, R., & Raake, A. (2023). Analysis of appeal for realistic AI-generated photos. IEEE Access, 11, 38999-39012. https://doi.org/10.1109/ACCESS.2023.3267968
Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: A workbook. Springer Nature. Haleem, A., Javaid, M., Qadri, M. A., Singh, R. P., & Suman, R. (2022). Artificial intelligence (AI) applications for marketing: a literature-based study. International Journal of Intelligent Networks. https://doi.org/10.1016/j.ijin.2022.08.005
Hausken, L. (2024). Photorealism versus photography. AI-generated depiction in the age of visual disinformation. Journal of Aesthetics & Culture, 16(1), 1-14. https://doi.org/10.1080/20004214.2024.2340787
He, A., Cai, Y., Cai, L., & Zhang, Y. (2021). Conversation, storytelling, or consumer interaction and participation? The impact of brand-owned social media content marketing on consumers’ brand perceptions and attitudes. Journal of Research in Interactive Marketing, 15(3), 419–440. https://doi.org/10.1108/JRIM-08-2019-0128
Hoeken, H., & Sinkeldam, J. (2014). The role of identification and perception of just outcome in evoking emotions in narrative persuasion. Journal of Communication, 64(5), 935-955. https://doi.org/10.1111/jcom.12114
Hsu, T., & Myers, S. L. (2023). Can we no longer believe anything we see? New York Times. https://www.nytimes.com/2023/04/08/business/media/ai-generated-images.html
Islam, T., Miron, A., Nandy, M., Choudrie, J., Liu, X., & Li, Y. (2024). Transforming digital marketing with generative AI. Computers, 13(7), 168. https://doi.org/10.3390/computer13070168
Jakesch, M., French, M., Ma, X., Hancock, J. T., & Naaman, M. (2019). AI-mediated communication: how the perception that profile text was written by AI affects trustworthiness. CHI, 1-13. https://doi.org/10.1145/32906005.3300469
Kaleel, A., & Alomari, M. S. (2023). Integrating artificial intelligence in public relations and media: a bibliometric analysis of emerging trends and influences. Iraqi Journal for Computer Science and Mathematics, 13-24. https://doi.org/10.52866/ijcsm.2024.05.01.002
Karinshak, E., Liu, S. X., Park, J. S., & Hancock, J. T. (2023). Working with AI to persuade: Examining a large language model’s ability to generate pro-vaccination messages. Proc. ACM Hum.-Comput. Interact, 7, 116-145. https://doi.org/10.1145/3579592
Kirk, C. P., & Givi, J. (2025). The AI-authorship effect: Understanding authenticity, moral disgust, and consumer responses to AI-generated marketing communications. Journal of Business Research, 186, 114984. https://doi.org/10.1016/j.jbusres.2024.114984
Kirkby, A., Baumgarth, C., & Henseler, J. (2023). To disclose or not disclose, is no longer the question – effect of AI-disclosed brand voice on brand authenticity and attitude. Journal of Product & Brand Management, 32, 1108-1122. https://doi.org/10.1108/JPBM-02-2022-3864
Kreps, S., McCain, R. M., & Brundage, M. (2020). All the news that’s fit to fabricate: AI-generated text as a tool of media misinformation. Journal of Experimental Political Science, 9, 104-117. https://doi.org/10.1017/XPS.2020.37
Ku?inskas, G. (2024). Negative effects of revealing AI involvement in products: mediation by authenticity and risk, moderation by trust in AI and familiarity with AI. 1-31. https://dx.doi.org/10.2139/ssrn.4805186
Lee, J. A., & Eastin, M. S. (2021). Perceived authenticity of social media influencers: scale development and validation. Journal of Research in Interactive Marketing, 15(4), 822-841. https://doi.org/10.1108/JRIM-12-2020-0253
Lu, H. (2025). Generative AI for vaccine misbelief correction: Insights from targeting extraversion and pseudoscientific beliefs. Vaccine, 54, 127018. https://doi.org/10.1016/just.vaccin.2025.127018
Lyu, Y., Wang, X., Lin, R., & Wu, J. (2022). Communication in human-AI co-creation: Perceptual analysis of paintings generated by text-to-image system. Applied Sciences, 12, 1-19. https://doi.org/10.3390/app122211312
Manoharan, A. (2024). Enhancing audience engagement through AI-powered social media Automation. World Journal of Advanced Engineering Technology and Sciences, 11(2), 150-157. https://doi.org/10.30574/wjaets.2024.11.2.0084
Mardhika, H. (2023). How the introduction of AI (Media Monitoring) tools affects the field of public relations. Jurnal Manajemen Bisnis, 10(2), 555-569. https://doi.org/10.33096/mb.v10i1.624
Martin, K., & Parmar, B. (2024). What firms must know before adopting AI: the ethics of AI transparency. 1-26. http://dx.doi.org/10.2139/ssrn.4207128
Murgai, A. (2018). Transforming digital marketing with artificial intelligence. International Journal of Latest Technology in Engineering, Management, & Applied Sciences, 7(4), 259-262.
Nair, K., & Gupta, R. (2020). Application of AI technology in modern digital marketing environment. World Journal of Entrepreneurship, Management, and Sustainable Development, 17(3), 318-328. https://doi.org/10.1108/WJEMSD-08-2020-0099
Neuman, W. R., & Guggenheim, L. (2011). The evolution of media effects theory: a six-stage model of cumulative research. Communication Theory, 21(2), 169-196. https://doi.org/10.1111/j.1468-2885.2011.01381.x
Niazi, M. A. K., Ghani, U., & Sadia, A. (2012). The emotionally charged advertisement and their influences on consumers’ attitudes. International Journal of Business and Social Science, 3(1), 307-320.
Noar, S. M., Palmgreen, P., Zimmerman, R. S., Lustria M. L. A., & Lu, Hung-Li. (2010). Assessing the relationship between perceived message sensation value and perceived message effectiveness: Analysis of PSAs from an effective campaign. Communication Studies, 61(1), 21-45. https://doi.org/10.1080/10510970903396477
Ohanian, R. (1990). Construction and validation of a scale to measure celebrity endorsers' perceived expertise, trustworthiness, and attractiveness. Journal of Advertising, 19(3), 39-52. https://doi.org/10.1080/00913367.1990.10673191
Pelet, J., Durrieu, F., & Lick, E. (2020). Label design of wines sold online: Effects of perceived authenticity on purchase intentions. Journal of Retailing and Consumer Services, 55. https://doi.org/10.1016/j.jretconser.2020.102087
Pragnashree, M. and Reddy, T. R. (2024), “Using conversational marketing to drive customer engagement: The era of AI and chatbots”, KIIT Journal of Management, Vol. 20 No. 1, pp. 80-92. https://doi.org/10.23862/kiit-parikalpana/2024/v20/i1/225093
Priester, J. R., & Petty, R. E. (2003). The influence of spokesperson trustworthiness on message elaboration, attitude strength, and advertising effectiveness. Journal of Consumer Psychology, 13(4), 408-421. https://doi.org/10.1207/S15327663JCP1304_08
Rahman, M. T., & Pial, T. (2020). Influence of rational and emotional appeals on purchasing through online: The case on social media. International Journal of Financial Research, 11(1), 34-42. https://doi.org/10.5430/ijfr.v11n1p34
Sardinha, T, B. (2024). AI-generated vs human-authored texts: A multidimensional comparison. Applied Corpus Linguistics, 4(1), 1-10. https://doi.org/10.1016/j.acorp.2023.100083
Shao, Z., Zhao, R., Yuan, S., Ding, M., & Wang, Y. (2022). Tracing the evolution of AI in the past decade and forecasting the emerging trends. Expert Systems with Applications, 209, 1-26. https://doi.org/10.1016/j.techfore.2019.119794
Song, M., Chen, H., Wang, Y., & Duan, Y. (2024). Can AI fully replace human designers? Matching effects between declared creator types and advertising appeals on tourists’ visit Intentions. Journal of Destination Marketing & Management, 32. https://doi.org/10.1016/j.jdmm.2024.100892
Tauheed, J., Shabbir, A., & Pervez, M. S. (2024). Exploring the role of artificial intelligence in digital marketing strategies. Journal of Business, Communication & Technology, 3(1), 54-65. http://doi.org/10.56632/bct.2024.3105
Toteva, M. (2023). Revolutionizing education: the transformative power of AI technologies in PR. Postmodernism Problems, 13(3), 307-320. https://doi.org/10.46324/PMP23033307
Tsai, W., Lun, D., Carcioppolo, N., & Chuan, C. (2021). Human versus chatbot: understanding the role of emotion in health marketing communication for vaccines. Psychology & Marketing, 38, 2377-2392. https://doi.org/10.1002/mar.21556
Yadav, N., Sinha, A., Jain, M., Agrawal, A., & Francis, S. (2024). Generation of images from text using AI. I. J. Engineering and Manufacturing, 14(1), 24-37. https://doi.org/10.5815/ijem.2024.01.03
Yang, J., & Jiang, M. (2021). Demystifying congruence effects in Instagram in-feed native ads: the role of media-based and self-based congruence. Journal of Research in Interactive Marketing, 15(4), 685-708. https://doi.org/10.1108/JRIM-03-2020-0048
Youn, S., & Jin, S. V. (2021). In AI we trust? The effects of parasocial interaction and technopian versus luddite ideological views on chatbot-based customer relationship management in the emerging feeling economy. Computers in Human Behavior, 119, 1-13. https://doi./10.1016/j.chb.2021.106721
Yzer, M., LoRusso, S., & Nagler, R. H. (2015). On the conceptual ambiguity surrounding perceived message effectiveness. Health Common, 30(2), 125-134. https://doi.org/10.1080/10410236.2014.974131
Zhang, J. J., Wang, Y. W., Ruan, Q., & Yang, T. (2024). Tourism Management Perspectives, 53, 1-13. https://doi.org/10.1016/j/tmp.2024.101279
Zhu, L., Lai, Y., Mou, W., Zhan, H., Lin, A., Qi, C. U., Yang, T., Xu, L., Zhang, J, & Luo, P. (2024). ChatGPT's ability to generate realistic experimental images poses a new challenge to academic integrity. Journal of Hematology & Oncology, 17(27), 1-3. https://doi.org/10.1186/s13045-024-01543-8
Ziakis, C., & Vlachopoulou, M. (2023). Artificial intelligence in digital marketing: insights from a comprehensive review. Information (Basel), 14(12), 664. https://doi.org/10.3390/info14120664
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Emma Salerno, Kate Salerno, Ruonan Zhang
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
