Installation

On your terminal run the following command to install ollama module.

pip install ollama

Pull and run the model of you choice. For this tutorial we are running Gemma2 2B model, since it is small and powerful

ollama pull gemma2:2b

Install Langchain-ollama module

pip install langchain-ollama

Import Model

For this experiment we will loading gemma2:2b using ollama and langchain model. The temperature is set to 0. Temperature near to 0 makes llm output more deterministic. For application executing certain task the temperature should be set near to 0. But for task like generating poem or movie scricpt the temperature should be set near to 1. The temperature near to 1 makes llm more creative and add ramdomness element to the process.

from langchain_ollama.llms import OllamaLLM
from langchain_core.prompts import ChatPromptTemplate
import json
from pprint import pprint

llm = OllamaLLM(model="gemma2:2b",
                temperature = 0)

Lets build LLM agent to translate one language to another.

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            """
            You are a helpful assistant that translates {input_language} to {output_language}. 
            Provides output in json format as language as key and outpus as value
            """,
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm

ai_message = chain.invoke(
    {
        "input_language": "English",
        "output_language": "Spanish, German, Korean",
        "input": "I love grilled chicken",
    }
)

pprint(ai_message)

('```json\n'
 '{\n'
 '  "English": "I love grilled chicken",\n'
 '  "Spanish": "¡Me encanta el pollo a la parrilla!",\n'
 '  "German": "Ich liebe gegrillte Hähnchen!",\n'
 '  "Korean": "닭갈비를 좋아해요!" \n'
 '}\n'
 '```')

The llm model response is in string format. Now, lets load the string response as json object.

# Replace and assign back to original content
ai_message = ai_message.replace("```json", "")
ai_message = ai_message.replace("```", "")

# Don't forget to convert to JSON as it is a string right now:
json_result = json.loads(ai_message)
pprint(json_result)

{'English': 'I love grilled chicken',
 'German': 'Ich liebe gegrillte Hähnchen!',
 'Korean': '닭갈비를 좋아해요!',
 'Spanish': '¡Me encanta el pollo a la parrilla!'}