Book a Call

AI and Creativity

AI Evaluation

AI Limitations

AI Chatbot

AI for Design

ARTICLE #32

Table of contents

1. Susceptibility to user bias

2. Inaccurate and unreliable algorithms

3. Absence of real-world context and perspective

4. Deficiency in creativity and original thought

5. Lack of emotional insight and empathy

6. Undermining human expertise and intuition

7. Ethical bias and considerations

8. Failure to grasp strategic objectives

9. Challenges in capturing brand voice

Final Thoughts

Evaluating creative work with AI: What are the possible risks?

Written by:

7 min read

Updated on: May 27, 2024

Samson Mosilily

Senior Regional Manager

African Market, Regional Management, Growth

Artificial intelligence, like ChatGPT or other large language models, can be downright fun to play with, and often it sounds convincingly human. But that slick veneer hides a thorny reality: AI isn’t exactly your best buddy when it comes to evaluating creative work. Sure, these models can help draft legal briefs, suggest marketing taglines, or even churn out books. Yet the moment you start leaning on them for deep insights or nuanced judgments in art, design, or conceptual thinking, you are basically asking your AI assistant to perform a job it wasn’t really built for.

Below are the reasons why relying on LLMs as your go-to “creative critic” might be a recipe for bland feedback and skewed results. From fundamental issues like data bias to the inherent lack of genuine creativity, we’ll explore the potential pitfalls you risk if you treat AI’s perspective as the ultimate expert opinion. After all, if real creativity were just about stringing words together, we’d all be Van Goghs and Shakespeares by now, but that’s not how the human spark works.

1. Susceptibility to user bias

AI chatbots, including popular Large Language Models (LLMs), respond heavily to prompts. If you instruct them to tear a piece of work apart, they’ll find flaws—whether those flaws are genuine or merely invented. Prompt them to praise the same piece, and suddenly they’ll gush about its brilliance.

This swing is particularly problematic when it comes to creativity, where nuance and context matter. Researchers have shown how easily GPT-style models can be nudged into biased or inaccurate answers just by adjusting the prompt (see various GPT-3 studies from leading academic sources, for instance). It’s a bit like asking someone who has never visited an art gallery to be your chief curator—handy in a pinch, but hardly a reliable authority.

2. Inaccurate and unreliable algorithms

AI Limitations often surface in the form of outdated data or oversimplified metrics. An AI that judges a text or design concept might look for patterns from its training data but overlook distinct cultural references or unusual style choices that don’t fit its set of rules. In a practical sense, imagine a brand that deliberately breaks grammar norms for impact. An AI tool might flag those stylistic decisions as errors. That means innovative ideas risk being labelled “incorrect.” And if you’re dealing with AI for Design, you might see bold or unconventional visuals get marked down because the algorithm “thinks” they’re mistakes.

3. Absence of real-world context and perspective

Though AI combs through mountains of data, it hasn’t lived a single day in the real world. It can’t attend an art exhibition, interpret local humour, or experience global events firsthand. This gap in genuine understanding becomes especially clear when evaluating witty references or subtle brand cues that rely on personal or cultural knowledge.

Take humour as an example. Sarcasm, irony, wordplay—all these require more than just dictionary definitions to interpret. Without that deeper grasp, the AI’s verdict may be shallow or off the mark.

4. Deficiency in creativity and original thought

Studies on creativity often define it through two lenses: “novelty” (offering something new) and “usefulness” (making that new idea beneficial). AI can churn out text that seems fresh, but beneath the surface, it’s typically recycling existing patterns.

When faced with boundary-pushing concepts—like a brand identity that defies category norms—an AI might view that approach as an error. Such “caution” from the AI could hamper bold moves that genuinely excite audiences.

Deficiency in Creativity and Original Thought

5. Lack of emotional insight and empathy

Yes, AI can scan text for sentiment, but it doesn’t truly feel anything. That means it can’t empathise with personal struggles or pick up on a brand’s emotional narrative. Where a creative strategist might sense the heartbreak or triumph behind a story and adjust accordingly, an AI simply scores or categorises it.

Crafting effective campaigns often hinges on intangible elements like gut feelings, cultural memory, and brand character. AI can’t replicate these human subtleties, making it ill-suited for any evaluation that hinges on emotional resonance.

6. Undermining human expertise and intuition

Too much reliance on AI Chatbot opinions might sideline professionals who’ve spent years honing their craft. Creative directors, copywriters, and brand strategists rely on experience that’s shaped by real-world results—far more nuanced than a dataset.

Ditching that hard-earned human expertise in favour of an algorithm’s quick take could not only flatten variety but also lessen the chances of a one-of-a-kind final product. Standardised evaluations might feel “safe,” but safe doesn’t always spark audience interest.

7. Ethical bias and considerations

Large AI models train on massive datasets that can contain ingrained biases about gender, race, and culture—you name it. If an AI offers creative feedback based on skewed data, it might inadvertently reinforce stereotypes. That’s hardly the path to fresh, inclusive brand concepts.

Moreover, an incorrectly trained AI might push you away from a campaign that resonates with a particular group, simply because the data it learned from wasn’t diverse or broad enough. We all know brand work can shape public attitudes, so these blind spots can become real hazards.

8. Failure to grasp strategic objectives

The brand expression usually follows a well-defined plan. Whether your project aims to attract a new audience or reposition your existing identity, there’s a strategic story behind each decision. AI often lacks an innate understanding of that high-level roadmap (careful not to use the taboo word “roadmap,” so let’s say “plan” instead).

Without context for the brand’s overall aims, an AI’s remarks may feel disconnected, making solid creative choices look like missteps.

9. Challenges in capturing brand voice

Brands invest countless hours refining their unique voice—how they speak, the tone they adopt, and the personality that comes through in writing and visuals. An AI can mimic many styles, but that’s often where it ends. It may unknowingly revert to a default style that aligns with its training data, missing the distinctive cadence or word choice that sets your brand apart.

In practice, an AI might misunderstand the difference between playful irreverence and unprofessional slang, leading to a critique that suggests “fixes” to elements that are, in fact, part of the brand’s signature.

Final Thoughts

Where creative judgment is involved, AI can certainly provide prompt ideas and interesting suggestions, but it cannot take the place of the final gatekeeper. ChatGPT and other such tools can churn out clever wordplay or flashy one-liners, but ethical considerations, brand tone, and cultural trends are not something that an AI can review with true understanding or empathy. Creativity needs a human hand, a person who can spot those nuanced emotional beats, maintain a brand narrative's consistency, and achieve the right balance of inspiration and strategy.

So, while AI is tempting (and undeniably useful in some stages of content creation), relying on it to green-light your boldest ideas or refine your brand’s style is a gamble. Ultimately, human expertise is what keeps creative work feeling fresh, relevant, and authentically connected to culture. Think of AI as a handy sidekick, not the creative director. By letting people handle the final call, agencies and brands can preserve the magic that makes their content truly resonate—and stay ahead of the curve as AI technology continues to evolve.