Questions tagged [ollama]
Ollama is a product for running Llama 2, Code Llama and other large language models.
ollama
139
questions
0
votes
0
answers
7
views
How to Immediately Cancel an Asyncio Task That uses the Ollama Python Library to generate an answer?
I'm using ollama to generate answers from large language models (LLMs) with ollama python api. I want to cancel the response generation by clicking the stop button. The problem is that the task ...
- 88
0
votes
1
answer
15
views
Does langchain with llama-cpp-python fail to work with very long prompts?
I'm trying to create a service using the llama3-70b model by combining langchain and llama-cpp-python on a server workstation. While the model works well with short prompts(question1, question2), it ...
0
votes
0
answers
20
views
How should I use Llama-3 properly?
I downloaded the Meta-Llama-3-70B-Instruct model using the download.sh and the url provided by Meta email, and this is all the files in the folder.
enter image description here
And when I tried to use ...
- 1
0
votes
1
answer
60
views
Slow Ollama API - how to make sure the GPU is used
I made a simple demo for a chatbox interface in Godot, using which you can chat with a language model, which runs using Ollama. Currently, the interface between Godot and the language model is based ...
- 6,452
0
votes
0
answers
28
views
How to use the godot-llama-cpp plugin
Godot newbye here. I made a simple chatbot demo (repo here) in Godot, which takes as input the text typed by a user and outputs the replies generated by a large language model running locally using ...
- 6,452
-1
votes
0
answers
92
views
Ollama erro (HTTPError: 404 Client Error: Not Found for url: http://localhost:11434/v1/chat/completions)
I'm trying to translate the results I scraped using Selenium.
But have been facing same issue while using ollama.
Here's the error in detail with my codes.
import os
import openai
import requests
...
2
votes
0
answers
47
views
langchain4j and Ollama - chat does not work because of uppercased role value
I am using Ollama v0.2.3 on Windows with tinyllama, locally installed, and langchain4j v0.32.0.
I followed a very simple example of sending a chat query to Ollama.
To my surprise I got back a very ...
- 452
0
votes
0
answers
77
views
How to stop Ollama model streaming
So I have this class that streams the response form a model:
from langchain_community.llms.ollama import Ollama
from app.config import (
LLM_MODEL_NAME,
MAX_LLM_INPUT_LENGTH,
...
- 4,367
0
votes
0
answers
28
views
doing embedding document in ollama with langchain always gets an error (400 Bad Request)
Always get error 400 Bad Request when i Trying to embedding document in ollama with langchain js.
here is my code
const embeddings = new OllamaEmbeddings({
model: "orca-mini",
baseUrl: &...
2
votes
1
answer
85
views
how to instantly terminate a thread? Using ollama python API with tkinter to stream a response from llama2
I'm using ollama to stream a response from llama2 large language model. The functionality I need is, when I click the stop button, it should stop the thread immediately. The code below works but the ...
- 88
-1
votes
1
answer
45
views
Ollama isnt using my gpu on a runpod.io pod
I am testing different AI models on runpod.io. One of those models is dolphin-mixtral:8x22b. I followed Runpod's tutorial for setting up the pod with Ollama: https://docs.runpod.io/tutorials/pods/run-...
- 1
0
votes
0
answers
21
views
How to configure ollama setup exe from its source code
I was required to install Ollama setup exe from the source code in windows
I found the steps as
Note: The Windows build for Ollama is still under development.
First, install required tools:
MSVC ...
0
votes
0
answers
59
views
The relationship between chunk_size, context length and embedding length in a Langchain RAG Framework
everyone. Currently I am working on a Langchain RAG framework using Ollama. I have a question towards the chunk size in the Document Splitter.
Now I decide to use qwen2:72b model as both embedding ...
- 149
0
votes
1
answer
56
views
Ollama Embeddings: Are the embeddings in the same order as the documents?
I’m using OllamaEmbeddings from langchain_community.embeddings (in Python) to generate embeddings of documents. I need to be absolutely sure that the embeddings are in same order as the documents that ...
- 1
1
vote
1
answer
210
views
LlamaParse not able to parse documents inside directory
Whenever I try to use LlamaParse I get an error that states the file_input must be a file path string, file bytes, or buffer object.
parser = LlamaParse(result_type="markdown")
...
