How to connect CrewAI to different LLMs (GPT4o, Groq, Llama3, Ollama) - Tutorial & LLM comparison

Published: 24 June 2024
on channel: Infinite Codes

3,553

114

I tested CrewAI with different LLMs (GPT4o, Groq, Llama3, Ollama) - FULL walkthrough tutorial & LLM comparison

In this tutorial, I will show you how to connect crewAI to different LLMs both local and in the cloud. I will then do a full performance and cost comparison for all the tested models. The tested LLMs are ChatGPT-3.5-turbo, ChatGPT-4-turbo, ChatGPT-4o, Groq with Meta’s Llama3 and Mixtral, as well as Llama3 locally with Ollama.

I will do a technology review for all the technologies used in the video and walk you through setting up the crew from cloning Joao Moura’s crewai-examples repository, environment and dependency setup to changing the code to use the different LLM backends. I will then run the code for the different LLMs and compare them in terms of runtime, cost and results

Links:
https://www.crewai.com/ CrewAI website
https://github.com/joaomdmoura/crewAI crewAI Github Repo
https://github.com/joaomdmoura/crewAI... crewAI Examples Repo

https://console.groq.com/ Groq API key
https://ollama.com/ Ollama
https://platform.openai.com/docs/over... OpenAI Platform (for OpenAI API key)
https://serper.dev/ SerperDev API key

My previous CrewAI tutorials:
• CrewAI Tutorial: Automate your Life w... general CrewAI tutorial
• I Automated my Instagram with AI Agen... CrewAI instagram Crew

Chapters

00:00 - Intro
01:08 - Technology Review: LLMs, CrewAI, AI Agents
03:46 - Set up the Crew: Clone the repo and change some code
12:20 - Set up environment and Dependencies, add API keys
17:15 - Comparing OpenAI LLMs (GPT-3.5, GPT-4, GPT-4o)
22:56 - Groq (with Llama3 and Mixtral)
29:40 - Run Local LLMs with Ollama
34:06 - Final Results & Cost comparison