# Building LLM Apps with Open AI Alternatives using LiteLLM

During the course of this bootcamp, many learners highlighted that OpenAI has now stopped giving free credits. As an easy option you can experiement by purchasing $5 credits. If that doesn't work, you don't need to worry as this sub-module helps you with options. 

 A great option to explore is the use of open source models or go for APIs offered by providers like Cohere, Replicate, Google, etc. This guide will show you how to integrate the LiteLLM connector with the Replicate API, providing a smooth transition.

## Step 1: Update and Install LiteLLM

First, let's ensure you have the latest version of LiteLLM, which will help you access the most up-to-date features and improve compatibility:

{% code overflow="wrap" %}

```python
# This command updates LiteLLM to the latest version.
!pip install -U litellm

# Alternatively, you can specify a minimum version like this: 
!pip install litellm>=1.37.20
```

{% endcode %}

## Step 2: Obtain and Set your API Key

* **Obtaining your API key**: Visit the [Replicate website](https://replicate.com/) to find your API key. It's typically located in the right sidebar on the documentation pages.
* **Setting your API key in Python**:

{% code overflow="wrap" %}

```python
import os 
os.environ["REPLICATE_API_KEY"] = "<YOUR_API_KEY>" 
# As a best practice, consider adding your API key to a `.env` file to keep it secure.
# Don't forget to load the env file with the `load_dotenv`!
```

{% endcode %}

## Step 3: Implementing LiteLLM in Python

Here's how you can set up LiteLLM for a simple chat application using Pathway. This example uses the Llama 3 model provided by Replicate:

{% code overflow="wrap" %}

```python
import pathway as pw 
from pathway.xpacks.llm.llms import LiteLLMChat, prompt_chat_single_qa 

# Initialize the LLM chat with the chosen model 
chat = LiteLLMChat(model="replicate/meta/meta-llama-3-8b")  # Example model 

# Prepare a query
query_table = pw.debug.table_from_markdown(''' 
txt 
Wazzup? 
''') 

# Generate a response from the LLM 
response = query_table.select(response=chat(prompt_chat_single_qa(query_table.query))) 

# Output the results to see the conversation in action 
print(response)
```

{% endcode %}

## **Integrating with Existing Applications**

If you're already using OpenAI LLMs in your apps, switching to LiteLLM is straightforward. Simply replace the model initialization line in your existing code:

```python
# Replace this for OpenAI: 
chat = OpenAIChat(model="gpt-4") 

# With this for Replicate:
chat = LiteLLMChat(model="replicate/meta/meta-llama-3-8b")
```

This change involves only a few lines, making it an efficient way to adapt to new LLM providers without overhauling your existing applications. If you have any questions or need further assistance, feel free to reach out via **#iit-delhi-bootcamp** channel on [Pathway Discord server](https://discord.com/invite/pathway).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://tryst-iit-delhi.gitbook.io/llm-course/hands-on-development/building-llm-apps-with-open-ai-alternatives-using-litellm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.