Book a Call
AI and Creativity
AI Evaluation
AI Limitations
AI Chatbot
AI for Design
ARTICLE #32
Table of contents
Evaluating creative work with AI: What are the possible risks?
AI and Creativity
AI Evaluation
AI Limitations
AI Chatbot
AI for Design
AI and Creativity
AI Evaluation
AI Limitations
AI Chatbot
AI for Design
Written by:
7 min read
Updated on: May 27, 2024
Samson Mosilily
Senior Regional Manager
African Market, Regional Management, Growth
Samson Mosilily
Senior Regional Manager
African Market, Regional Management, Growth
Artificial Intelligence (AI) is no longer just churning out quick blog posts or chatbot replies. We’ve all heard about how AI for Design, AI Evaluation, and AI Chatbot solutions might help legal experts, assist doctors, or even lend a hand in mental health. Impressive? Absolutely. But handing over creative judgement to an algorithm can be an entirely different kettle of fish.
Below, we’ll explore some of the key reasons why relying on AI for creative critique—or letting an AI chatbot weigh in on your brand strategy—may lead you astray.
Artificial Intelligence (AI) is no longer just churning out quick blog posts or chatbot replies. We’ve all heard about how AI for Design, AI Evaluation, and AI Chatbot solutions might help legal experts, assist doctors, or even lend a hand in mental health. Impressive? Absolutely. But handing over creative judgement to an algorithm can be an entirely different kettle of fish.
Below, we’ll explore some of the key reasons why relying on AI for creative critique—or letting an AI chatbot weigh in on your brand strategy—may lead you astray.
1. Susceptibility to user bias
1. Susceptibility to user bias
AI chatbots, including popular Large Language Models (LLMs), respond heavily to prompts. If you instruct them to tear a piece of work apart, they’ll find flaws—whether those flaws are genuine or merely invented. Prompt them to praise the same piece, and suddenly they’ll gush about its brilliance.
This swing is particularly problematic when it comes to creativity, where nuance and context matter. Researchers have shown how easily GPT-style models can be nudged into biased or inaccurate answers just by adjusting the prompt (see various GPT-3 studies from leading academic sources, for instance). It’s a bit like asking someone who has never visited an art gallery to be your chief curator—handy in a pinch, but hardly a reliable authority.
AI chatbots, including popular Large Language Models (LLMs), respond heavily to prompts. If you instruct them to tear a piece of work apart, they’ll find flaws—whether those flaws are genuine or merely invented. Prompt them to praise the same piece, and suddenly they’ll gush about its brilliance.
This swing is particularly problematic when it comes to creativity, where nuance and context matter. Researchers have shown how easily GPT-style models can be nudged into biased or inaccurate answers just by adjusting the prompt (see various GPT-3 studies from leading academic sources, for instance). It’s a bit like asking someone who has never visited an art gallery to be your chief curator—handy in a pinch, but hardly a reliable authority.
2. Inaccurate and unreliable algorithms
AI Limitations often surface in the form of outdated data or oversimplified metrics. An AI that judges a text or design concept might look for patterns from its training data but overlook distinct cultural references or unusual style choices that don’t fit its set of rules.
In a practical sense, imagine a brand that deliberately breaks grammar norms for impact. An AI tool might flag those stylistic decisions as errors. That means innovative ideas risk being labelled “incorrect.” And if you’re dealing with AI for Design, you might see bold or unconventional visuals get marked down because the algorithm “thinks” they’re mistakes.
AI Limitations often surface in the form of outdated data or oversimplified metrics. An AI that judges a text or design concept might look for patterns from its training data but overlook distinct cultural references or unusual style choices that don’t fit its set of rules.
In a practical sense, imagine a brand that deliberately breaks grammar norms for impact. An AI tool might flag those stylistic decisions as errors. That means innovative ideas risk being labelled “incorrect.” And if you’re dealing with AI for Design, you might see bold or unconventional visuals get marked down because the algorithm “thinks” they’re mistakes.
3. Absence of real-world context and perspective
Though AI combs through mountains of data, it hasn’t lived a single day in the real world. It can’t attend an art exhibition, interpret local humour, or experience global events firsthand. This gap in genuine understanding becomes especially clear when evaluating witty references or subtle brand cues that rely on personal or cultural knowledge.
Take humour as an example. Sarcasm, irony, wordplay—all these require more than just dictionary definitions to interpret. Without that deeper grasp, the AI’s verdict may be shallow or off the mark.
Though AI combs through mountains of data, it hasn’t lived a single day in the real world. It can’t attend an art exhibition, interpret local humour, or experience global events firsthand. This gap in genuine understanding becomes especially clear when evaluating witty references or subtle brand cues that rely on personal or cultural knowledge.
Take humour as an example. Sarcasm, irony, wordplay—all these require more than just dictionary definitions to interpret. Without that deeper grasp, the AI’s verdict may be shallow or off the mark.
4. Deficiency in creativity and original thought
Studies on creativity often define it through two lenses: “novelty” (offering something new) and “usefulness” (making that new idea beneficial). AI can churn out text that seems fresh, but beneath the surface, it’s typically recycling existing patterns.
When faced with boundary-pushing concepts—like a brand identity that defies category norms—an AI might view that approach as an error. Such “caution” from the AI could hamper bold moves that genuinely excite audiences.
Studies on creativity often define it through two lenses: “novelty” (offering something new) and “usefulness” (making that new idea beneficial). AI can churn out text that seems fresh, but beneath the surface, it’s typically recycling existing patterns.
When faced with boundary-pushing concepts—like a brand identity that defies category norms—an AI might view that approach as an error. Such “caution” from the AI could hamper bold moves that genuinely excite audiences.
5. Lack of emotional insight and empathy
Yes, AI can scan text for sentiment, but it doesn’t truly feel anything. That means it can’t empathise with personal struggles or pick up on a brand’s emotional narrative. Where a creative strategist might sense the heartbreak or triumph behind a story and adjust accordingly, an AI simply scores or categorises it.
Crafting effective campaigns often hinges on intangible elements like gut feelings, cultural memory, and brand character. AI can’t replicate these human subtleties, making it ill-suited for any evaluation that hinges on emotional resonance.
Yes, AI can scan text for sentiment, but it doesn’t truly feel anything. That means it can’t empathise with personal struggles or pick up on a brand’s emotional narrative. Where a creative strategist might sense the heartbreak or triumph behind a story and adjust accordingly, an AI simply scores or categorises it.
Crafting effective campaigns often hinges on intangible elements like gut feelings, cultural memory, and brand character. AI can’t replicate these human subtleties, making it ill-suited for any evaluation that hinges on emotional resonance.
6. Undermining human expertise and intuition
Too much reliance on AI Chatbot opinions might sideline professionals who’ve spent years honing their craft. Creative directors, copywriters, and brand strategists rely on experience that’s shaped by real-world results—far more nuanced than a dataset.
Ditching that hard-earned human expertise in favour of an algorithm’s quick take could not only flatten variety but also lessen the chances of a one-of-a-kind final product. Standardised evaluations might feel “safe,” but safe doesn’t always spark audience interest.
Too much reliance on AI Chatbot opinions might sideline professionals who’ve spent years honing their craft. Creative directors, copywriters, and brand strategists rely on experience that’s shaped by real-world results—far more nuanced than a dataset.
Ditching that hard-earned human expertise in favour of an algorithm’s quick take could not only flatten variety but also lessen the chances of a one-of-a-kind final product. Standardised evaluations might feel “safe,” but safe doesn’t always spark audience interest.
7. Ethical bias and considerations
Large AI models train on massive datasets that can contain ingrained biases about gender, race, and culture—you name it. If an AI offers creative feedback based on skewed data, it might inadvertently reinforce stereotypes. That’s hardly the path to fresh, inclusive brand concepts.
Moreover, an incorrectly trained AI might push you away from a campaign that resonates with a particular group, simply because the data it learned from wasn’t diverse or broad enough. We all know brand work can shape public attitudes, so these blind spots can become real hazards.
Large AI models train on massive datasets that can contain ingrained biases about gender, race, and culture—you name it. If an AI offers creative feedback based on skewed data, it might inadvertently reinforce stereotypes. That’s hardly the path to fresh, inclusive brand concepts.
Moreover, an incorrectly trained AI might push you away from a campaign that resonates with a particular group, simply because the data it learned from wasn’t diverse or broad enough. We all know brand work can shape public attitudes, so these blind spots can become real hazards.
8. Failure to grasp strategic objectives
The brand expression usually follows a well-defined plan. Whether your project aims to attract a new audience or reposition your existing identity, there’s a strategic story behind each decision. AI often lacks an innate understanding of that high-level roadmap (careful not to use the taboo word “roadmap,” so let’s say “plan” instead).
Without context for the brand’s overall aims, an AI’s remarks may feel disconnected, making solid creative choices look like missteps.
The brand expression usually follows a well-defined plan. Whether your project aims to attract a new audience or reposition your existing identity, there’s a strategic story behind each decision. AI often lacks an innate understanding of that high-level roadmap (careful not to use the taboo word “roadmap,” so let’s say “plan” instead).
Without context for the brand’s overall aims, an AI’s remarks may feel disconnected, making solid creative choices look like missteps.
9. Challenges in capturing brand voice
Brands invest countless hours refining their unique voice—how they speak, the tone they adopt, and the personality that comes through in writing and visuals. An AI can mimic many styles, but that’s often where it ends. It may unknowingly revert to a default style that aligns with its training data, missing the distinctive cadence or word choice that sets your brand apart.
In practice, an AI might misunderstand the difference between playful irreverence and unprofessional slang, leading to a critique that suggests “fixes” to elements that are, in fact, part of the brand’s signature.
Brands invest countless hours refining their unique voice—how they speak, the tone they adopt, and the personality that comes through in writing and visuals. An AI can mimic many styles, but that’s often where it ends. It may unknowingly revert to a default style that aligns with its training data, missing the distinctive cadence or word choice that sets your brand apart.
In practice, an AI might misunderstand the difference between playful irreverence and unprofessional slang, leading to a critique that suggests “fixes” to elements that are, in fact, part of the brand’s signature.
Final Thoughts
From uncertain algorithms to biases and a lack of emotional awareness, using AI for creative evaluations can be a dicey move. Sure, AI can generate endless content and handle tasks like grammar checks or fact-finding with remarkable speed. But it shouldn’t be the final word on creative judgement, especially when big-picture branding decisions are at stake.
From uncertain algorithms to biases and a lack of emotional awareness, using AI for creative evaluations can be a dicey move. Sure, AI can generate endless content and handle tasks like grammar checks or fact-finding with remarkable speed. But it shouldn’t be the final word on creative judgement, especially when big-picture branding decisions are at stake.
More news
Work with us
Click to copy
work@for.co
FOR® Industries
- FOR® Brand. FOR® Future.
We’re remote-first — with strategic global hubs
Click to copy
Helsinki, FIN
info@for.fi
Click to copy
New York, NY
ny@for.co
Click to copy
Miami, FL
mia@for.co
Click to copy
Dubai, UAE
uae@for.co
Click to copy
Kyiv, UA
kyiv@for.co
Click to copy
Lagos, NG
lagos@for.ng
Copyright © 2024 FOR®
Work with us
Click to copy
work@for.co
FOR® Industries
We’re remote-first — with strategic global hubs
Click to copy
Helsinki, FIN
hel@for.co
Click to copy
New York, NY
ny@for.co
Click to copy
Miami, FL
mia@for.co
Click to copy
Dubai, UAE
uae@for.co
Click to copy
Kyiv, UA
kyiv@for.co
Click to copy
Lagos, NG
lagos@for.ng
Copyright © 2024 FOR®