Best Large Language Model (LLM) of 2024: Data-Backed Study

AI large language models (LLMs) are taking the internet by storm, making it easier than ever before to create content and images, as well as source information for your daily queries. 

But what’s the best AI language model?

After all, if you’re forking out $20 a month for an AI tool, you may as well choose the best one, right?

Well, we did the research, and we got the results! 

My team and I analyzed three of the most popular AI language tools for response quality and content ideation. Check out what we learned below and find out which LLM is best. 

Key Takeaways

• We asked Gemini, ChatGPT, and Copilot two questions (one to measure accuracy and one to measure strategic advice) to determine the best response. 
• We measured each LLM’s response against nine criteria on a scale from 0 to 3, with a total score of 27 points.  
• Copilot was the best performer overall, though Gemini and ChatGPT were able to provide fairly comprehensive responses in certain circumstances as well.
• There are dozens of use cases for AI large language models, including topic creation, article outlines, feedback, and quality assurance. 
• I don’t recommend you let AI LLMs write your content unless you are willing to edit and fact-check them heavily.

Our Methodology

To discover which AI language model is the best from a marketing and SEO perspective, we asked Gemini, Copilot, and ChatGPT two carefully crafted questions — one to determine the model’s accuracy and the other to assess its strategic capabilities. 

We then ranked the quality of each response across nine factors on a scale from 0 to 3.

Those factors were:

1. Accuracy. Is the response factually correct?
2. Level of detail. Does the response provide enough context?
3. Readability. Does the response make sense, and is it written in layperson’s terms? 
4. Length. Is the response too long or too short?
5. Links to resources. Are there links to relevant resources?
6. Follow-up questions. How well does the LLM answer follow-up questions?
7. Response time. How long does it take to respond?
8. Authenticity. Does it sound human or AI?
9. Digital accessibility. Does the tool assist individuals with disabilities by providing text-based interactions, which can be easier to navigate than other interfaces?

Why did we go this route? Ryan Velez, Sr. Content Production Lead at NP Digital, explains

“Our goal with constructing these questions was making sure that we could test as many aspects of comprehension as possible for the LLM. How well can it parse information to get the most accurate and up-to-date results? If we want to go deeper, how well can it process follow-up results?”

Each response received a score out of 27, with a maximum of 52 points available. The model with the highest total score was crowned the best AI large language model.

Know Your LLMs

Before we dive into the results, let’s take a moment to introduce our competitors. We compared the three leading large language models from Google, Microsoft, and OpenAI. 

ChatGPT

OpenAI’s ChatGPT is the original AI LLM. Launched in November 2022, it became the fastest-growing app of all time, reaching 100 million monthly active users after just two months.

ChatGPT started off as a generative text model that let users ask questions and carry out other text-based activities. Today, it has several other AI models, like DALL·E and Whisper, that let users create images and transcribe speech into text.

ChatGPT is free to use. A premium version, ChatGPT Plus, costs $20 per month. 

Gemini

Gemini, formerly known as Bard, is Google’s AI LLM that launched in March 2023. The platform can do almost anything ChatGPT can do, including answering questions and creating images. 

The model’s multimodal capabilities are particularly impressive and allow users to create video, audio, and code on the same platform. 

Gemini is free to use. The premium version, called Gemini Advanced, costs $19.99 per month. 

Copilot

Copilot is Microsoft’s AI large language model that launched in February 2023. The tool uses Microsoft’s Prometheus model, which is an extension of OpenAI’s GPT-4 foundational large language model.

Microsoft has integrated Copilot into several products, including the Edge browser, Bing Search, and the Microsoft 365 suite.

You can use Copilot for free, but you’ll need to pay an additional monthly fee if you want the Microsoft 365 integration. 

It’s important to note that all of these LLMs have similar limitations. They are prone to hallucinations, which is when they fabricate inaccurate information, cannot reason, and have limited long-term coherence.  

Wondering what a hallucination looks like in practice? Imagine that I ask ChatGPT when Michaelangelo painted the Sistine Chapel, and it answers 1930. That’s a hallucination since Michaelangelo painted the Sistine Chapel in the early 1500s.  

With that in mind, let’s dig into the nuts and bolts of how these different models perform.

Question 1: Accuracy-Based Question

To start, we asked an accuracy-based marketing question:

Can you tell me who Neil Patel is and what kind of marketing services he provides?

We also asked two follow-up questions:

• What are some blog topics on neilpatel.com?
• Who are some of NPD’s SEO clients?

Copilot was the clear winner here, as you can see from the results table below:

Copilot’s response, which you can see below, was the best by a pretty long way.

“Certainly! Neil Patel is a prominent figure in the world of digital marketing. Let me introduce you to his background and the services he offers: 

Background: 

• Neil Patel is a New York Times Bestselling author and a highly influential personality on the web. 
• Forbes recognizes him as one of the top 10 marketers, and Entrepreneur Magazine credits him with creating one of the 100 most brilliant companies. 
• He has received accolades such as being named a top 100 entrepreneur under the age of 30 by President Obama and a top 100 entrepreneur under the age of 35 by the United Nations. 

Marketing Services: 

• Content Marketing: Neil Patel’s team specializes in creating compelling content that attracts traffic, gets shared, and earns valuable links. 
• Digital PR: They elevate brand reputation and exposure through strategic Digital PR solutions. 
• Influencer Marketing: Neil Patel harnesses the power of influencers to amplify brand reach and impact. 

NP Digital: 

• Neil Patel’s agency, NP Digital, offers a wide range of marketing services: 
• Search Engine Optimization (SEO): Optimizing websites to improve organic search visibility. 
• Paid Search Marketing: Effective paid strategies with clear ROI. 
• Social Media Marketing: Leveraging social platforms for brand growth. 
• Content Marketing: Crafting epic content that attracts links and drives traffic. 
• Data Analytics and Insight: Providing actionable insights for informed decision-making. 
• Programmatic Advertising: Targeted ad placements across digital channels. 

NP Accel: 

• NP Accel is another arm of Neil Patel’s agency, focusing on helping companies grow through social media, video, SEO content, and more. 
• In summary, Neil Patel’s expertise spans various facets of digital marketing, making him a sought-after resource for businesses aiming to succeed online “

Not only was it accurate and detailed, but it also provided several links to my website and knew who my clients were. Copilot also has several accessibility options, including a voice-to-text option, that make it the easiest tool for people with disabilities to use. 

Gemini’s response was okay, but it lacked detail. 

“Neil Patel is a prominent figure in the digital marketing world, known for his expertise in various online marketing aspects. Here’s a breakdown of his profile and services: 

Who is Neil Patel? 

• A New York Times bestselling author and recognized influencer by publications like The Wall Street Journal and Forbes. 
• Co-founder of NP Digital, a leading global digital marketing agency. 
• Previously co-founded companies like Crazy Egg and Kissmetrics, establishing a strong track record in the marketing technology space. 
• Credited with creating one of the 100 most brilliant companies by Entrepreneur Magazine and recognized as a top entrepreneur under 30 by both President Obama and the United Nations. 

What kind of marketing services does he provide? 

While Neil Patel himself doesn’t directly provide individual marketing services, his expertise extends through various channels: 

• NP Digital: His co-founded agency, NP Digital, offers a broad range of digital marketing services, including: 

• Search Engine Optimization (SEO) 
• Conversion Rate Optimization (CRO) 
• Content Marketing 
• Paid Media Management 
• Social Media Marketing 
• Content and education: Neil Patel actively shares his knowledge through various mediums: 

• His personal blog, covering a wide range of digital marketing topics 
• Educational programs and resources 

Overall, Neil Patel is a respected figure in the digital marketing landscape, offering both direct agency services through NP Digital and valuable insights and education through his content and resources. 

Source: https://www.expatden.com/need-help-in-thailand/digital-marketing/ “

It also did a poor job of answering follow-up questions. Providing blog topics was outside of its capabilities, and it was unable to disclose my agency’s SEO clients due to “confidentiality reasons.” That’s despite many being publicly available on my agency website.  

ChatGPT’s response was the worst of the lot. 

“Neil Patel is a well-known entrepreneur, marketer, and author in the digital marketing industry. He is recognized for his expertise in online marketing and has made significant contributions to the field. Neil Patel has co-founded companies like Crazy Egg, Hello Bar, and KISSmetrics, all of which provide various tools and services related to digital marketing and analytics. 

As of my last knowledge update in January 2022, Neil Patel is also a prolific blogger and shares valuable insights on his blog (neilpatel.com) where he covers topics such as SEO (Search Engine Optimization), content marketing, social media, and other aspects of digital marketing. He has a strong online presence and is widely followed for his educational content. 

Neil Patel offers consulting services, conducts webinars, and speaks at conferences to share his knowledge and expertise in digital marketing. It’s important to note that the specific services and focus areas of Neil Patel may evolve over time, so for the most up-to-date information, it’s recommended to check his official website or other recent sources.”

The response was limited, outdated and didn’t include links to additional resources. It did a better job answering follow-up questions than Gemini, but it doesn’t have the digital accessibility features of the other two tools, like the ability to tab between replies and voice-to-text recognition. 

Winner: Copilot

Question 2: Strategic-Advice Based Question

To determine the quality of the strategic advice given by each large language model, we asked the following question: 

I am a small business that sells window treatments and operates in 25 cities around the United States. Can you give me recommendations on what my local search engine optimization (SEO) strategy should be?

Please explain why this would be a good strategy and how I can beat my competitors in Google’s search engine results page.

Copilot was the winner again, but Gemini and ChatGPT performed a lot better than before. 

Copilot’s response was the most detailed, accurate, and actionable. 

Certainly! As a small business selling window treatments across 25 cities in the United States, optimizing your local SEO strategy is crucial for attracting potential customers and outperforming competitors. Let’s dive into actionable recommendations: 

Google My Business (GMB): 

• Claim and optimize your GMB listing: Ensure your business details (address, phone number, website, hours of operation) are accurate. Regularly update posts, photos, and respond to reviews. 
• Local citations: Consistent NAP (Name, Address, Phone) information across online directories and platforms helps improve local rankings. 

Local Keyword Research: 

• Identify local keywords related to window treatments and your service areas. Use tools like Google Keyword Planner or SEMrush. 
• Optimize your website content with these keywords, especially on location-specific pages. 

Localized Content: 

• Create city-specific landing pages on your website. Highlight local projects, customer testimonials, and unique offerings. 
• Write blog posts about local events, trends, and how your window treatments enhance local living spaces. 

Online Reviews and Reputation Management: 

• Encourage satisfied customers to leave positive reviews on Google and other platforms. 
• Respond promptly to both positive and negative reviews. Show your commitment to customer satisfaction. 

Local Backlinks: 

• Build local backlinks from reputable websites in your service areas. Collaborate with local bloggers, home improvement websites, and community organizations. 
• Sponsor local events or participate in community activities to earn backlinks. 

Mobile Optimization: 

• Ensure your website is mobile-friendly. Many local searches happen on mobile devices. 
• Optimize page load speed for better user experience. 

Schema Markup: 

• Implement local business schema markup on your website. It helps search engines understand your business details and service areas. 

Social Media Engagement: 

• Be active on social media platforms relevant to your business. Share local content, engage with followers, and promote special offers. 
• Social signals impact local SEO. 

Competitor Analysis: 

• Study your competitors’ local SEO strategies. Identify gaps and opportunities. 
• Analyze their backlink profiles, content, and GMB presence. 

Track and Measure: 

• Use tools like Google Analytics and Google Search Console to monitor local traffic, keyword rankings, and user behavior. 
• Adjust your strategy based on data insights. 

Why This Strategy Works: 

• Local Relevance: Focusing on local keywords and content ensures you’re relevant to users in each city. 
• Trust Signals: Consistent NAP, positive reviews, and local backlinks build trust with search engines. 
• User Experience: Mobile optimization and fast-loading pages enhance user experience. 
• Visibility: GMB optimization and local citations improve visibility in local searches. 

Beating Competitors: 

• Quality Content: Create valuable, localized content that answers user queries. 
• Local Backlinks: Earn high-quality backlinks from local sources. 
• Consistent Effort: Regularly update your website, engage on social media, and monitor performance. 

Remember, local SEO is an ongoing process. Stay committed, adapt to changes, and provide exceptional service to stand out in Google’s search results!”

It provided specific instructions on improving local SEO, such as using tools like Semrush and Google Search Console. 

It also included several local SEO and marketing links, which would be helpful for finding out more, as well as a link to a company that provides local SEO services. 

ChatGPT’s response was a lot better than its response for the first question. 

Certainly! For a small business operating in multiple cities like yours, a local search engine optimization (SEO) strategy is crucial to improve visibility and attract potential customers in each location. Here are some recommendations for your local SEO strategy: 

Optimize Your Website for Local Keywords: 

• Include city-specific keywords in your website content, meta tags, and headings. For example, use phrases like “window treatments in [City Name]” to target local search queries. 

Create Location-Specific Landing Pages: 

• Develop individual landing pages for each city you operate in. These pages should include relevant information about your products and services in that particular location. Highlight local testimonials, projects, or partnerships to make the content more location-specific. 

Claim and Optimize Google My Business (GMB) Listings: 

• Claim your Google My Business listings for each of the 25 cities. Ensure that your business name, address, and phone number (NAP) are accurate and consistent across all listings. Add high-quality images and provide detailed information about your window treatments. 

Encourage Customer Reviews: 

• Positive reviews play a significant role in local SEO. Encourage satisfied customers to leave reviews on your Google My Business profiles. Respond to reviews, both positive and negative, to show engagement and build trust. 

Local Citations and Directory Listings: 

• Ensure your business is listed accurately on relevant local directories and citation sites. Consistent NAP information across the web signals credibility to search engines. Include details such as business hours, website URL, and a brief description. 

Mobile Optimization: 

• Ensure your website is mobile-friendly. Many local searches happen on mobile devices, and Google gives preference to mobile-optimized sites in its rankings. 

Local Link Building: 

• Build local backlinks by reaching out to local businesses, community organizations, and partners for collaboration. Participate in local events, sponsorships, or partnerships that can lead to local press coverage and backlinks. 

Social Media Presence: 

• Maintain an active and localized presence on social media platforms. Share content related to your products, showcase local installations, and engage with the local community. Social signals can influence local search rankings. 

Monitor and Analyze Local SEO Performance: 

• Use tools like Google Analytics, Google Search Console, and local SEO tools to monitor the performance of your local SEO efforts. Analyze data to understand what strategies are working and where adjustments may be needed. 

Adapt to Google Algorithm Updates: 

• Stay informed about Google’s algorithm updates and adjust your strategy accordingly. Google’s algorithms are continually evolving, and staying up-to-date ensures that your local SEO efforts remain effective. 

By implementing a comprehensive local SEO strategy, you increase the likelihood of appearing prominently in Google’s search engine results pages (SERPs) for relevant local searches. Be consistent with your efforts across all locations, and focus on providing valuable, locally relevant content and services to your customers. Regularly update and refine your strategy based on performance metrics and changes in the digital landscape to maintain a competitive edge. 

It provided good detail, but the lack of links and digital accessibility options let it down. It also wasn’t quite as actionable as Copilot’s response, says Jeff Durante, Sr. SEO Strategist & Local SEO Team Lead at NP Digital:

“Overall, I think the information touches on important general elements to local but misses the ‘how’, while also containing some information that can be misinterpreted into utilizing spammy tactics and/or a lack of depth of optimization for on-page content and GBPs.”

We ranked Gemini last for this question. 

Here are some recommendations for your local SEO strategy, considering you operate in 25 cities across the United States: 

1. Claim and optimize your Google My Business (GMB) profiles for each city: 

• This is crucial for local businesses. It allows you to manage your online presence on Google Maps and Search results. 
• Ensure your business name, address, phone number, website URL, and hours of operation are accurate and consistent across all profiles. 
• Encourage customers to leave reviews on your GMB profiles, as positive reviews increase trust and ranking. 
• Add high-quality photos showcasing your work and window treatments. 
• Utilize GMB posts to announce promotions, new products, or events in each city. 

2. Local Keyword Research: 

• Research keywords specific to each city you operate in. Use tools like Google Keyword Planner or Ahrefs to find relevant keywords with search volume and lower competition. 
• Target keywords that include city names, like “window blinds [city name]” or “custom curtains [city name].” 
• Integrate these local keywords strategically throughout your website content, including page titles, meta descriptions, and service pages. 

3. Build local citations: 

• Citations are online mentions of your business name, address, and phone number (NAP) across various directories and websites. 
• Submit your business information to local directories, industry-specific websites, and online yellow pages. 
• Ensure NAP consistency across all platforms. 
• Encourage partners and vendors to mention your business on their websites with a link back to yours. 

4. Create location-specific website pages: 

• Develop dedicated pages for each city you serve. Include city-specific information like address, phone number, and a unique description mentioning local landmarks or neighborhoods. 
• Optimize these pages with relevant local keywords. 

5. Engage in community events and sponsorships: 

• Build local brand awareness and backlinks by participating in community events, sponsoring local sports teams, or partnering with local businesses. 
• This demonstrates your involvement in the community and can lead to mentions and links from local news websites or blogs. 

6. Encourage online reviews: 

• Positive online reviews are crucial for local SEO. Encourage satisfied customers to leave reviews on your GMB profiles, Facebook page, Yelp, and other relevant platforms. 
• Respond to all reviews, both positive and negative, in a professional and timely manner. 

Why is this a good strategy? 

• This strategy focuses on local relevance and trust signals, which are crucial factors in Google’s local search algorithm. By implementing these tactics, you increase your local visibility, improve your chances of appearing in local search results, and build trust with potential customers searching for window treatments in your cities. 

Beating competitors: 

• It’s difficult to guarantee surpassing all competitors, but this strategy helps you stand out. By consistently implementing these tactics, you demonstrate a strong local presence, build trust with local customers, and gain valuable backlinks, all contributing to better rankings in Google’s local search engine results pages. 

Remember, SEO is an ongoing process. Regularly monitor your performance, analyze competitor strategies, and adapt your approach based on results for continuous improvement. 

It just didn’t have the same level of detail as the other responses, although its follow-up answer was impressive.

Winner: Copilot

Which LLM Was Most Effective?

I think it’s pretty obvious that Copilot was the best AI LLM for answering our marketing and SEO-related questions. 

It provided particularly detailed and nuanced answers, regularly linked to relevant resources, and responded in a way that would make it easy to copy and paste the answer with only minimal editing required. 

Copilot’s advice on local SEO particularly impressed Nikki Brandemarte, Sr. SEO Strategist & Local SEO Specialist at NP Digital: 

“I like how this response touches on the importance of publishing related blog content. The other two responses seem to focus more so on creating location landing pages but publishing relevant informational content is also important in order to show your expertise in a particular industry.”

Copilot was also the best AI large language model from a digital accessibility standpoint.

How To Most Effectively Use LLMs

So you know which AI LLM is the best. Now, let’s cover the best ways to use Copilot and other AI tools:

• Content ideation. The best AI tools for content ideation can provide dozens of awesome blog posts and whitepaper topics for your writers. 
• Structuring articles. AI large language models are great tools for creating a quick blog article outline. They can help you ensure you have the right points in the right order. 
• Getting feedback. Want to make sure your blog post has enough examples, and you’ve tied up all the loose ends? Have an AI tool provide feedback on your draft.
• Quality assurance. These tools are also great at checking spelling, grammar, and style. 

This all boils down to the prompts you create, though. Austin Cosler, Managing Editor at NP Digital, explains, “Crafting effective prompts for LLMs is an art. The key lies in keeping them concise and focused. Instead of cramming in multiple questions, ask one clear and specific question at a time. Providing well-defined context and examples (known as shots) can significantly improve a model’s understanding. Don’t forget the power of Determine Appropriate Response (DARE) prompts — these are special prompts that leverage the LLM itself to decide if its response aligns with the mission or purpose you give it.”

The one thing I wouldn’t have an AI large language model do just yet is create a draft of my content. It’s not yet at the same level as human writers, and there are several ways you can tell AI content apart if you know what to look for. These include:

• Vague and lengthy introductions
• A lack of specific or personal examples
• Similar formatting patterns and using the same phrases. 
• Overuse of buzzwords or emojis. 

But hey, if you’re happy to heavily edit your AI LLM’s first draft, then be my guest. 

FAQs

Conclusion

So there you have it. If you’re looking for the best AI large language model, look no further than Microsoft Copilot. If you want a tools that provides comprehensive and accurate answers that need minimal editing, responds well to follow-up questions, and provides sources, Copilot is the perfect choice. 

With that said, as LLMs evolve, who knows what model will stay on top. Scott Zimmerman, Associate Director of Technical SEO at NP Digital, explains that “the future advancements of large language models (LLMs) will be substantially driven by their ability to integrate and contextualize information from a diverse array of file formats. Currently, LLMs are primarily text-based, but expanding their capabilities to analyze and understand content from various formats—such as images, videos, and audio—could revolutionize their utility.”
So, let’s put these insights to work. Create your own blog posts with AI LLM using the these prompts I recommend or learn how to improve your SEO with ChatGPT. 

See How My Agency Can Drive More Traffic to Your Website

• SEO - unlock more SEO traffic. See real results.
• Content Marketing - our team creates epic content that will get shared, get links, and attract traffic.
• Paid Media - effective paid strategies with clear ROI.

Book a Call

Are You Using Google Ads? Try Our FREE Ads Grader!

Stop wasting money and unlock the hidden potential of your advertising.

• Discover the power of intentional advertising.
• Reach your ideal target audience.
• Maximize ad spend efficiency.

Get your free Analysis