How to Use Groq's FREE API in Your Streamlit App

Groq’s LPU inference engine delivers some of the fastest text generation you can get from a cloud API, and the free tier gives you real access with no credit card. If you want to build an AI-powered Streamlit app without paying for API calls, this is the fastest path I know.

The original version of this article (March 2024) used Mistral models on Groq. That no longer works. Groq deprecated all Mistral models by July 2025. This updated guide uses Llama 3.1 8B Instant instead: same free tier, same Groq speed, and a model that’s still actively supported. The core integration pattern hasn’t changed, so most of what you learn here applies regardless of which Groq model you pick.

You can have a working AI-powered Streamlit app running in under 15 minutes.

What changed since 2024: Mistral deprecation and new models

Here’s the timeline:

• Feb 2024: Groq launches with Mixtral and Mistral models available via API.
• March 2025: mixtral-8x7b-32768 is deprecated and shut down.
• July 2025: The last Mistral model on Groq (mistral-saba-24b) is deprecated. No Mistral models remain on the platform.

If you’re specifically looking for Mistral, see the Mistral alternative section below. Mistral AI runs their own API with a free tier.

Current free-tier models (July 2026)

For this tutorial, I default to llama-3.1-8b-instant. It’s fast, cheap (if you ever hit the paid tier), and good enough for most prototyping work. If you need higher quality output, llama-3.3-70b-versatile is the upgrade, at the cost of lower rate limits. Check the best open-source LLMs for a broader comparison of what’s available today.

Groq also now offers a Developer Tier (pay-as-you-go, credit card required) with roughly 10x the free-tier limits and access to their Batch API. For prototyping and small apps, the free tier is plenty.

If you’re scaling up and need to compare model costs across providers, see cheapest AI models for agent workflows.

Prerequisites: what you need to get started

If you’re completely new to Python and AI development, start with getting started with AI programming first.

Setting up the Groq Python SDK

Install the Groq SDK and python-dotenv for managing your API key:

pip install groq python-dotenv

Create a .env file in your project directory:

echo 'GROQ_API_KEY=gsk_your_api_key_here' > .env

Replace gsk_your_api_key_here with the key from console.groq.com/keys.

Now write a quick verification script to confirm everything works:

import os
from dotenv import load_dotenv
from groq import Groq

load_dotenv()

client = Groq(api_key=os.environ.get("GROQ_API_KEY"))

completion = client.chat.completions.create(
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
    max_tokens=50,
)

print(completion.choices[0].message.content)

Run it:

python verify.py

How to use the Groq API with Python

The core pattern is straightforward: create a client, send a chat completion request, handle the streamed response.

Here’s the request lifecycle:

Python Script → Groq SDK → Groq API (LPU) → Llama 3.1 8B → Streamed Response

The Groq SDK handles HTTP connection management, retries (2 automatic retries on transient errors), and timeout configuration. You don’t need to manage any of that yourself for basic usage.

Breaking down the API parameters

completion = client.chat.completions.create(
    model="llama-3.1-8b-instant",
    messages=[
        {
            "role": "system",
            "content": "You are a YouTube expert who writes engaging titles."
        },
        {
            "role": "user",
            "content": "Install WordPress on Docker"
        }
    ],
    temperature=0.5,
    max_tokens=1024,
    top_p=1,
    stream=True,
)

• model: The model ID. llama-3.1-8b-instant is the fast free-tier default.
• messages: A list of message objects. The system role sets behavior; the user role provides your prompt.
• temperature (0-2): Controls randomness. 0.5 gives focused but not rigid output. Use 0 for deterministic responses, 1+ for more creative ones.
• max_tokens: Maximum tokens in the response. 1024 is plenty for 10 YouTube titles. The old article used 5640, which was way too high.
• top_p: Alternative to temperature for nucleus sampling. 1 means no filtering. Leave it at 1 unless you have a specific reason to change it.
• stream: True streams chunks as they generate. This gives much better perceived latency in a UI.

Complete Python script with error handling

import os
from dotenv import load_dotenv
from groq import Groq

load_dotenv()

client = Groq(api_key=os.environ.get("GROQ_API_KEY"))

try:
    completion = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[
            {
                "role": "system",
                "content": "You are a YouTube expert creator who likes to write engaging titles for a keyword. You will provide 10 attention-grabbing YouTube titles on keywords specified by the user."
            },
            {
                "role": "user",
                "content": "Install WordPress on Docker"
            }
        ],
        temperature=0.5,
        max_tokens=1024,
        top_p=1,
        stream=True,
    )

    for chunk in completion:
        content = chunk.choices[0].delta.content
        if content:
            print(content, end="")

except groq.RateLimitError:
    print("Rate limit hit. Wait a minute and try again.")
except groq.APIConnectionError:
    print("Network issue. Check your internet connection.")
except groq.APIStatusError as e:
    print(f"API error {e.status_code}: {e.response}")
except Exception as e:
    print(f"Unexpected error: {e}")

Run this and you’ll see YouTube titles streamed to your terminal in real time. The speed is noticeable. Groq’s LPU generates tokens significantly faster than most cloud APIs.

Build a Streamlit app with Groq and Llama 3.1

Streamlit turns any Python function into a web UI with minimal boilerplate. If you’ve tried other Python UI frameworks, you know Streamlit trades customization for speed of development. For a quick AI demo app, that’s the right tradeoff. For a deeper comparison, see Streamlit vs NiceGUI.

Install Streamlit:

pip install streamlit

Create app.py:

import os
from dotenv import load_dotenv
import streamlit as st
from groq import Groq

load_dotenv()

def get_groq_completions(user_content):
    client = Groq(api_key=os.environ.get("GROQ_API_KEY"))

    completion = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[
            {
                "role": "system",
                "content": "You are a YouTube expert creator who likes to write engaging titles for a keyword. You will provide 10 attention-grabbing YouTube titles on keywords specified by the user."
            },
            {
                "role": "user",
                "content": user_content
            }
        ],
        temperature=0.5,
        max_tokens=1024,
        top_p=1,
        stream=True,
    )

    result = ""
    for chunk in completion:
        content = chunk.choices[0].delta.content
        if content:
            result += content
    return result

def main():
    st.title("YouTube Title Generator")
    st.write("Powered by Groq LPU + Llama 3.1 8B")

    user_content = st.text_input("Enter the keyword for YouTube titles:")

    if st.button("Generate Titles"):
        if not user_content:
            st.warning("Please enter a keyword before generating titles.")
            return

        with st.spinner("Generating titles..."):
            try:
                generated_titles = get_groq_completions(user_content)
                st.success("Titles generated successfully!")
                st.markdown("### Generated Titles:")
                st.text_area("", value=generated_titles, height=200)
            except Exception as e:
                st.error(f"Error: {e}")

if __name__ == "__main__":
    main()

Run it:

streamlit run app.py

The app opens at http://localhost:8501. Enter a keyword, click Generate Titles, and you’ll see results appear in the text area.

A few things worth noting about the code above:

• st.spinner instead of st.info: shows a loading animation, which is better UX than a static message.
• Error handling in the UI: st.error() shows errors inline rather than crashing the app.
• load_dotenv(): loads the .env file automatically. For Streamlit Cloud deployment, use st.secrets instead (covered in the deployment section).

Choosing the right Groq model for your app

The default llama-3.1-8b-instant works well for most prototyping. But Groq offers several models, and the right choice depends on your use case.

model="llama-3.1-8b-instant",

model="llama-3.3-70b-versatile",

model="openai/gpt-oss-20b",

Handling rate limits and errors on the free tier

The original article had zero error handling. On the free tier, you will hit rate limits eventually, especially if you’re testing rapidly or building something people actually use. Here’s what you need to know.

Free-tier limits per model

RPM = requests per minute, RPD = requests per day, TPM = tokens per minute, TPD = tokens per day.

For a personal demo app, these limits are generous. For anything with real users, you’ll outgrow them fast.

Error handling pattern for Streamlit

import groq

def get_groq_completions(user_content):
    client = Groq(api_key=os.environ.get("GROQ_API_KEY"))

    try:
        completion = client.chat.completions.create(
            model="llama-3.1-8b-instant",
            messages=[
                {"role": "system", "content": "You are a YouTube expert..."},
                {"role": "user", "content": user_content}
            ],
            temperature=0.5,
            max_tokens=1024,
            stream=True,
        )

        result = ""
        for chunk in completion:
            content = chunk.choices[0].delta.content
            if content:
                result += content
        return result

    except groq.RateLimitError:
        return "⏳ Rate limit hit. Please wait about a minute and try again."
    except groq.APIConnectionError:
        return "🔌 Network error. Check your internet connection."
    except groq.APIStatusError as e:
        return f"⚠️ API error ({e.status_code}): Please try again later."

The key thing: show the user a message they can act on. A cryptic stack trace in a Streamlit app helps nobody.

What the rate limit headers tell you

Groq’s API response includes headers you can inspect programmatically:

• x-ratelimit-remaining-requests: Requests left in the current window
• x-ratelimit-remaining-tokens: Tokens left in the current window
• x-ratelimit-reset-tokens: Time until the token limit resets

For a simple Streamlit app, you don’t need to parse these. But if you’re building something more complex (queue-based processing, multi-user apps), monitoring these headers lets you implement proactive backoff instead of waiting for a 429 error.

Deploying your Streamlit app

Once your app works locally, you’ll want to deploy it. Two main paths:

GROQ_API_KEY = "gsk_your_api_key_here"

groq
python-dotenv
streamlit

FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

EXPOSE 8501
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

services:
  app:
    build: .
    ports:
      - "8501:8501"
    environment:
      - GROQ_API_KEY=${GROQ_API_KEY}
    restart: unless-stopped

# Set your API key
export GROQ_API_KEY=gsk_your_api_key_here

# Build and run
docker compose up -d

Mistral alternative: using Mistral’s own API

If you arrived at this article looking for Mistral specifically, Mistral AI runs their own API at console.mistral.ai. The free “Experiment” tier gives you access to their current models including Mistral Small and Mistral Nemo.

Quick example using the mistralai Python SDK:

pip install mistralai python-dotenv

import os
from dotenv import load_dotenv
from mistralai import Mistral

load_dotenv()

client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"))

completion = client.chat.complete(
    model="mistral-small-latest",
    messages=[
        {"role": "system", "content": "You are a YouTube expert who writes engaging titles."},
        {"role": "user", "content": "Install WordPress on Docker"}
    ],
    temperature=0.5,
    max_tokens=1024,
)

print(completion.choices[0].message.content)

The pattern is similar to Groq. The main difference: Mistral’s API runs on their own infrastructure, so you won’t get Groq’s LPU speed advantage. But the models themselves are competitive, especially for European data residency requirements.

Conclusion and next steps

Groq’s free API plus Streamlit gives you a working AI app in minutes with no credit card and no infrastructure to manage. The integration is straightforward: install the SDK, send chat completions, display results.

Compared to the 2024 version of this article:

• The model changed from Mixtral to Llama 3.1 8B Instant (Mixtral was deprecated)
• Error handling is now included (the original had none)
• Rate limits are documented (the original didn’t mention them)
• Deployment options cover both free Community Cloud and Docker on a VPS

Where to go from here:

• Try different models — swap llama-3.1-8b-instant for llama-3.3-70b-versatile and compare output quality
• Add chat history — use st.session_state to maintain conversation context across interactions
• Structured outputs — use Groq’s response_format with JSON schema to get clean data instead of parsing text (useful for the YouTube title generator — imagine getting back a JSON array of titles)
• Build something more complex — try building an AI research squad with Streamlit or build a full AI agent
• Run models locally — if privacy matters, you can run LLMs locally with Ollama and skip the cloud API entirely