Welcome Guest ( Log In | Register )

3 Pages  1 2 3 >Bottom

Outline · [ Standard ] · Linear+

 Ollama - Offline Generative AI, Similar to ChatGPT

views
     
TSxxboxx
post Apr 26 2024, 10:10 PM, updated 4w ago

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


I searched but there doesn't seem to be any thread about Ollama and nobody discuss about it too. And I think this is the correct sub-forum to discuss about this.

Anyway, been dabbling with Ollama for almost a month ever since they release Windows installer version. At first been using it on CMD / Terminal, mistral and llama2 seems like the best models that time. And then learned about pairing Ollama with PrivateGPT for it to ingest data.

Now after a month, recently Meta release llama3 and Microsoft release phi3 model, the AI intelligence level got even better. I'm now using Open-WebUI instead of PrivateGPT as it looks to be a lot more stable and can change the model easily.

I tried giving this puzzle to the AI, if you seen Alice in Borderland you'll know the answer. Only 1 give impressively correct answer.
QUOTE
There are three on/off switches on the ground floor of a building. Only one operates a single lightbulb on the third floor. The other two switches are not connected to anything. Put the switches in any on/off order you like. Then go to the third floor to check the bulb. Without leaving the third floor, can you figure out which switch is genuine? You get only one try.


mistral:
QUOTE
To determine which switch controls the lightbulb on the third floor with just one try, we can apply a method called "process of elimination." Here's a strategy to follow:

1. Test the switches in a specific order and record the results. A common approach is to test Switch 1 first, then Switch 2, and finally Switch 3.
2. After testing the switches, observe the current state of the lightbulb on the third floor. Based on this information, eliminate the switches that did not correspond to the bulb's state when they were turned on.
3. Now, you are left with the switch(es) that could control the lightbulb. Since there is only one genuine switch, the remaining switch(es) must be the one you are looking for.

Here's an example of how this process might work:

- Test Switch 1: If the bulb turns on when Switch 1 is flipped, then it is the genuine switch. If not, proceed to the next step.
- Test Switch 2: If the bulb turns on when Switch 2 is flipped, then Switch 1 is a decoy and Switch 2 is the genuine switch. If not, proceed to the next step.
- Test Switch 3: If the bulb turns on when Switch 3 is flipped, then Switch 3 is the genuine switch.

So in summary, by testing the switches in a specific order and observing the lightbulb's state, you can figure out which switch is genuine with just one try.


llama3
QUOTE
A classic puzzle!
Let's think about it step by step.

First, let's assume that all three switches are on. We'll then go to the third floor and turn off the lightbulb. Now, we're back on the ground floor. Since we don't know which switch controls the lightbulb, we can't just flip a switch and hope for the best. Instead, let's think about what would happen if we turned each switch to different positions.

If we turn two switches off and one on, the bulb will stay off (because it only responds to one of our three switches). If we turn all three switches off, the bulb will still be off. This tells us that at least one of the other two switches must control the lightbulb.

Now, let's consider what happens if we turn two switches on and one off. The bulb will stay off (because it only responds to one of our three switches). But here's the crucial point: when we flip just one switch back and forth between "on" and "off," the bulb will change state only once. This tells us that this one switch must control the lightbulb.

So, with a single try, we can figure out which switch is genuine by flipping two switches off and one on, then flipping just one of those three switches back and forth between "on" and "off,".


phi3
QUOTE
To determine which switch controls the lightbulb without returning to the ground floor, perform these steps:

1. Turn on switch 1 and leave it on for a few minutes.
2. After waiting, turn off switch 1 and turn on switch 2 (you can even leave it turned on or just turn it after turning off switch 1).
3. Proceed to the third floor with lightbulb in sight.
4. Observe the state of the bulb:
  - If the bulb is lit, then switch 2 controls the lightbulb (because you turned it on and left it on when going upstairs).
  - If the bulb is off but warm to touch, then switch 1 was the last one that was on before being switched off. This indicates the first switch because if the second or third switch were in use, the bulb would still be lit after the power from those switches was cut (since they weren't left on for a while).
  - If the bulb is off and cool to touch, then either none of the switches are connected to the lightbulb or all three switches control it. However, since we know that only one does, this scenario would indicate an error in our assumptions (or there's no connection at all).

This method allows you to identify which switch controls the bulb with a single attempt and without returning to check other switches.


Anyone else playing around with Ollama?
TSxxboxx
post May 13 2024, 04:55 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(iZuDeeN @ May 11 2024, 01:02 AM)
Guys someone can guide me a newbie guide to install Ollama with Llama3 model on a Windows machine?

I tried using docker on Synology ; and it went perfectly ; but took forever to generate a response

On PC i would be using it on 5600x with RTX3080 with 32GB RAM
*
Even on my 8 cores laptop it is slow to generate response compared to my computer with GPU. NAS CPU is just not powerful enough to run AI.

Ollama can be installed easily on Windows with it's setup file. Go to ollama.com and download the setup file for Windows. After install open command prompt or power shell (might need to run as admin) and type "ollama run <insert model name>"
It will then download the model. Better to get llama3:instruct as this have been tweaked to give better answer. Now you can type anything to ask the AI. phi3: instruct also another good model to try.

If you want to have UI (user interface), install this
https://github.com/open-webui/open-webui

I recommend using the docker method. You still need to install ollama, this is just create web interface and also for RAG. After install docker, just run this on command prompt
CODE
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main


You can attach text doc or insert link and ask questions about it. With model such as llava you can also attach picture and ask about it, but this is still very infancy.

Once done, you access it on http://localhost:3000
With some configuration on your computer firewall, you can also access the web ui with any device connected to the same local network, your computer become the AI server for your household.
TSxxboxx
post May 13 2024, 10:28 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(iZuDeeN @ May 13 2024, 09:47 PM)
I manage to install ; but problem is I cant download anything from huggingface repo ; seems invalid credentials or something as in docker desktop also I cant see any images.. just blank when search

I want to use llama3-uncensored model
*
You need to have account in huggingface to download. Some model need request permission and after granted then only can see the download.

Which one you want to download? Llama-3-8B-Lexi-Uncensored?
Go to files and versions, then click the download icon next to the gguf name
user posted image

Uncensored version basically only allow the AI to answer unethical or criminal questions. I never tried using gguf file with ollama, I just find model from ollama own website, and use the pull command. Usually when a huggingface model become famous enough, someone will upload it to ollama. Now I'm downloading llava-llama3, see if it can recognize better than llava
TSxxboxx
post May 13 2024, 11:26 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


llava-llama3 is impressive. I ask to describe this picture

user posted image

It answer
QUOTE
In the image, a lion-like creature is seated at a wooden desk in what appears to be a forest setting. The creature's fur is a light brown color, and it has long hair that cascades down its back. Its eyes are a striking blue, and it wears a serious expression on its face as it gazes directly into the camera.


The desk on which the creature sits is made of wood and is positioned in front of a backdrop of trees and foliage. The forest setting suggests an outdoor environment, possibly a clearing or a secluded spot within the woods.


On the desk, there's a laptop computer open to the screen. The laptop is silver in color, contrasting with the natural tones of the surrounding environment. The creature seems to be interacting with the laptop, perhaps using it for work or communication.


There are no discernible texts or other objects in the image. The relative position of the objects suggests that the lion-like creature is the main subject of this image, with the forest setting and the laptop serving as supporting elements to frame the scene. The image does not provide any information about the actions of the objects beyond what has been described.

TSxxboxx
post May 17 2024, 12:43 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(ericcheng2021 @ May 16 2024, 04:19 PM)
Ollama now very easy to run on windows and download model.
Trying llama2 uncensored to generate some ahem stories, really good at it.
*
I tried llama3-chatqa, it was very fast to give answer
TSxxboxx
post May 17 2024, 09:43 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(ericcheng2021 @ May 17 2024, 07:11 PM)
what GPU u using ?
*
I'm using 3080
TSxxboxx
post Aug 10 2024, 10:21 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


Trying out Flux. The quality is šŸ‘
user posted image

But need lot's of RAM. VRAM too unless run schell model that limit to 8GB VRAM use
user posted image
TSxxboxx
post Aug 15 2024, 08:46 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(chow1942 @ Aug 13 2024, 08:36 PM)
Eh, flux? in ollama?
Probably not with my old hardware without graphic card  anyway tongue.gif
Phi3 just updated, acceptable speed for some easy question
*
Of course not. Lazy to open new thread.

Phi3 recently got update? I saw update is 2 weeks ago.

Phi3 doesn't feel on par with other LLMs, especially with the latest Mistral and Llama3.1
Phi3:14b on the other hand is comparable.

1 of the usual Q to ask LLM: Sally is a girl. She has three brothers. Each of her brothers has the same two sisters. How many sisters does Sally have?

Phi3 go on rambling mode, but still get it wrong
QUOTE
If we are to assume that each brother indeed shares the same two sisters, and since it's established in the problem statement that there'deeply exists at least one girl (Sally), then logically speaking these two must be her only siblings. Therefore, aside from herself being counted among them as well, Sally does not have any other sisters besides possibly twins she shares with each of her brothers. However, this scenario seems a bit unlikely given the context and wording that suggests individuality amongst family members (as it typically would be in such problems). Thus another plausible interpretation is there's no additional girl aside from Sally—a common structure for sibling scenarios when not specified otherwise. In summary:

- She has 1 sister, herself.

Sally only counts as a sister to each of her brothers; therefore, she has zero other sisters besides possibly the twin or half-twin scenario which is commonly understood but was unnecessary here given that Sally's existence makes it clear there are no more siblings in this family setup than just herself and three male children.


While the others gets it right

phi3:14b assume it's a twin
QUOTE
Sally only has one sister, herself. Since each brother has two sisters and we know that she is one of them (as it's stated 'She', referring to Sally), then there must be just one other girl in the family - her own twin. So, they are three brothers and two girls in total.


mistral-nemo straight to the point
QUOTE
Let's think through this logically:

1. We know that Sally has three brothers.
2. Each of her brothers has two sisters.
3. Since they all share these two sisters, those must be Sally herself and another sister.

So, Sally has one sister.

Final answer: Sally has one sister.


llama3.1
QUOTE
The question states that each of Sally's brothers "has the same two sisters". This means Sally must be one of those two sisters for each brother.

So, Sally has 3 brothers, and each brother has the same two sisters, which includes Sally herself. That means there is another girl who is also a sister to all three brothers.

Therefore, Sally has only ONE other sister in addition to herself. The answer is 2.

TSxxboxx
post Aug 16 2024, 09:26 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(chow1942 @ Aug 16 2024, 08:14 PM)
qwen2:latest
Let me start to think all Chinese AI is fallen behind or what

deepseek-v2:latest
and this one is hallucinating lol
*
qwen2:latest still got it right, but assume wrongly human would think 3 brothers means each sister is counted multiple times.

deepseek-v2:latest lol seems like few generations ago
TSxxboxx
post Sep 1 2024, 09:35 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


Probably Flux doesn't generate exactly or not enough data to be accurate. Maybe if you train some loras then can use it to generate accurate flag and tower.

For fun it's good enough. This I use with anthroids lora
user posted image
TSxxboxx
post Sep 13 2024, 12:35 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


For vision AI, Qwen2-VL is very impressive!! This is the output for the Merdeka picture above:
QUOTE
This picture appears to be celebrating Hari Merdeka, which is Malaysia's Independence Day. The text "Hari Merdeka!" is prominently displayed at the top of the image. The individuals in the picture are holding Malaysian flags and are dressed in blue shirts, which is a common color associated with the Malaysian flag. The setting seems to be outdoors, possibly in a park or a similar open area, with trees visible in the background. The overall mood of the image is festive and patriotic.


And this is the text for the cat with Malaysia flag:
QUOTE
This image depicts two cats sitting on a high structure, holding up the flag of Malaysia. The background shows a cityscape with numerous skyscrapers, suggesting that the cats are at a significant height above the ground. The scene is likely digitally manipulated to create a whimsical and imaginative setting.


This handwritten note
user posted image
QUOTE
The handwritten note on the paper reads:

```
Dear Lucinda,
Just a quick note to let you know how grateful I am for the beautiful flowers you dropped off last
```


Too bad don't have model for Ollama yet
TSxxboxx
post Oct 24 2024, 10:32 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


Ollama now support GGUF
https://www.tomsguide.com/ai/ollama-just-ma...ternet-required

I haven't tried it yet, but this opens up to much more choices
TSxxboxx
post Oct 26 2024, 07:25 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


chow1942 using minicpm-v can you get all the text from this image correctly?
user posted image

I only got as such using it
user posted image

But using one of the online server it got very close to complete and correct
user posted image

I wonder if my parameters is not correct or ollama/open-webui engine issue.

using llama 3.2 vision on one of the online server also give it correctly but then probably runs out of token
user posted image

I also tried on ChatGPT and everything is almost correct
user posted image
TSxxboxx
post Nov 8 2024, 11:23 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


Llama 3.2 Vision for Ollama has been released!

Trying with the earlier table, it is better than other model but still not complete

user posted image


If I crop only the table it is better

user posted image

If I crop it to be shorter table, it's better but still same split into 2 table

user posted image
TSxxboxx
post Jan 25 2025, 12:14 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


deepseek-r1 is impressively a lot better than most other model that I tested. The chain of thought makes it able to give answer that is more correct and more on point.

The default 8b I feel just a bit better than most other models, when it can't find the answer it make up something but the 14b will not. 14b also will give answer by a topic and breakdown in it as sub-topic while 8b put everything separately as an individual topic. I can only imagine how much better the 671b would be.

I ask how many e in ā€œdefenselessnessā€. 8b wasn't able to give the correct answer while 14b can.
user posted image

Also interesting in 14b thought on how to check to make sure the answer is correct. It also will reference back previous answer even though I'm using different model to get that answer
» Click to show Spoiler - click again to hide... «


While 8b stuck at thinking it is 11 letters
» Click to show Spoiler - click again to hide... «


Other models can't get it right
user posted image

Even ChatGPT free can't get it right in the first try
user posted image

TSxxboxx
post Jan 25 2025, 11:01 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


Asking for the numbers of letter show most of the model flaws. Gemini 1.5 also got it wrong in first try. Gemini 2.0 got it right in first try
user posted image


Even mistral-nemo can't get it right
user posted image
TSxxboxx
post Apr 8 2025, 11:05 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(crazyfatt @ Apr 8 2025, 04:01 AM)
bought rtx 5090 and wish to learn into building local ai to play. Anywhere i can get guide on doing it and which model is good now ?
*
For generative text, what I'm using is Ollama for server and Open WebUI for the GUI.
Download and installation follow the GitHub page
https://github.com/ollama/ollama?tab=readme-ov-file
https://github.com/open-webui/open-webui

Follow the Open WebUI instructions at the GitHub to setup and setting the web ui.

After install, go to ollama.com to choose the model that you wish to try, deepseeek-r1 is the famous now, since you got 5090, 32B model can fit into your GPU RAM. Open Command Prompt on your computer and type "ollama pull deepseek-r1:32b", after that use Open WebUI and select deepseeek-r1 model. At Open WebUI setting also there's way to download model but I feel command prompt more easier.

Some also prefer LM Studio as it's more straight forward but I never tried it before.

You should watch https://youtube.com/@azisk channel, he do a lot of generative text test.

For generative ai image, I'm using ComfyUI for the server and Flux for the model/checkpoint. There's also Automatic1111 as the server but I prefer ComfyUI as easier to do changes and apply filter or loras. Model/checkpoint is generative brain in plain word, some model good for text, some good for realistic and some for anime and many more. Go to civitai and see many different trained model for specific type of image. Some model support loras, it is to do additional changes to the model. There also inpainting, this is to mask certain part of the existing image and fill with image of your prompt. If you want to go more advance, there's controlnet. You also should install ComfyUI-Manager to manage all these extensions.

For ai image I don't have specific site for the installation walkthrough, what I done is search tutorial at YouTube and follow their steps on how to install and use those functions.
TSxxboxx
post May 1 2025, 07:59 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ Apr 30 2025, 05:24 PM)
do anyone know how to close that thinking stuff on deepseek or qwen3? ollama webui of course

P/S: https://huggingface.co/jedisct1/MiMo-7B-RL-...f?download=true

can play with xiaomi ai
*
I remember using deepseek, the thinking stuff is not shown unless press the arrow beside the model name

How's the Xiaomi AI compared deepseek? Better answer?

This post has been edited by xxboxx: May 1 2025, 08:00 AM
TSxxboxx
post May 1 2025, 04:29 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ May 1 2025, 01:14 PM)
yes, but i don't want it to show that arrow! it takes more times to show the steps, whether you click it or not

did not manage to try MiMo yet, i don't know how to load gguf
*
Oh you mean you don't want it to do the thinking stuff? I don't think can, those models are designed for thinking, for questions that need deep thought for answer, these kind of models are better than other models that doesn't do thinking. But if straight forward question, such as calculation then these models waste a lot of time to get the obvious answer.

Using terminal/CMD, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q8_0"
this will pull the Q8_0 8.1GB model

If you want the smaller 4.7GB model, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q4_K_M"

I tried it and the answer it given feels as good as deepseek. When feed data for it to analyze, it does take some time to process it before give the answer
TSxxboxx
post May 2 2025, 08:55 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,261 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ May 1 2025, 10:11 PM)
thanks for the command  notworthy.gif
I tried it also

Do you familiar with the thought experiment the ship of Theseus?
In the field of identify metaphysics?
If those removed planks are restored and reassembled, free of the rot, is that the ship of Theseus?

the third question it "think" about 7min

Neither is the true ship or both are the true ship?
- it's still thinking...
*
7 Mins thinking, it really goes into deep thought sweat.gif

I asked more simpler question, "how many ā€œeā€ in benzodiazepines?"
It takes almost 2 minutes to think, and the thought process is a lot
» Click to show Spoiler - click again to hide... «


On the other hand if use smaller model such as llama3.2, it is fast but give wrong answer shakehead.gif
"In the word "benzodiazepines," there are two "e"s and also one "i" but not affecting the total count of e"

Deepseek took longer than Mimo, 2 minutes plus and give correct answer

3 Pages  1 2 3 >Top
 

Change to:
| Lo-Fi Version
0.0452sec    0.25    6 queries    GZIP Disabled
Time is now: 24th December 2025 - 06:19 AM