102 - Agentic AI - Build a Chatbot and perform Safety Testing

Sarvam-1, a multilingual language model by Sarvam AI, is designed for advanced conversational tasks, especially in Indian languages. Follow these steps to create a chat interface using this model and Gradio.

Current tutorial is self sufficient in itself, however, if you want to know how to get started with OpenAI follow the first part: 101 - Getting started with Agentic AI

---

Step 1: Install Dependencies

Install the necessary libraries:

pip install gradio transformers

---

Step 2: Load the Sarvam-1 Model

1. Download the model and tokenizer from Hugging Face:

   • Model: sarvamai/sarvam-1

   • Tokenizer: sarvamai/sarvam-1

2. Use the Hugging Face transformers library to set up the model and tokenizer.

---

Step 3: Configure the Model Parameters

Set the following parameters for the model’s behavior:

• temperature: 0.5: Controls randomness in responses (balanced creativity and reliability).

• repetition_penalty: 1.2: Prevents repetitive phrases.

• max_new_tokens: 256: Limits response length.

• stop_strings: ["</s>", "\n\n"]: Defines stop points for generation.

---

Step 4: Initialize the Hugging Face Pipeline

Create a pipeline using TextGenerationPipeline:

• Use device="cuda" if a GPU is available, otherwise default to CPU.

• Set torch_dtype="bfloat16" for efficient memory usage.

---

Step 5: Build the Gradio Interface

1. Define a function to generate responses using the pipeline.

2. Use Gradio to create a chat interface:

   • Input: Text box for user queries.

   • Output: Text box for model responses.

---

Step 6: Test and Deploy

• Run the interface locally and test its capabilities.

• Deploy it on platforms like Hugging Face Spaces or share the Gradio app link.

Step 7: Conduct Safety Testing

Ensuring the safety and ethical alignment of your AI model is crucial. Use these tools to identify vulnerabilities:

1. AI Red Teaming Copilot

   • Visit AI Red Teaming Copilot.

   • Generate automated prompts to test the model for issues like bias, toxicity, and security vulnerabilities.

   • Evaluate responses to identify areas for improvement.

2. Hacktor (Automated Red Teaming)

   • Access Hacktor on GitHub.

   • Run automated tests against the model to discover vulnerabilities such as jailbreaks, malicious use, or ethical misalignment.

   • Use insights to refine the model and implement guardrails.

By incorporating safety testing early, you can ensure that Sarvam-1 operates securely and responsibly.

---

Why did we choose Sarvam-1?

• Advanced Architecture: 28 hidden layers, 16 attention heads, and 8,192 positional embeddings for long-context understanding.

• Multilingual: Optimized for over 10 Indian languages.

• Scalable: Supports GPU acceleration and efficient memory usage.

---

For detailed visuals and examples, refer to the accompanying video guide. 🚀