Welcome Guest ( Log In | Register )

5 Pages < 1 2 3 4 > » Bottom

Outline · [ Standard ] · Linear+

 Ollama - Offline Generative AI, Similar to ChatGPT

views
     
TSxxboxx
post Sep 13 2024, 12:35 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


For vision AI, Qwen2-VL is very impressive!! This is the output for the Merdeka picture above:
QUOTE
This picture appears to be celebrating Hari Merdeka, which is Malaysia's Independence Day. The text "Hari Merdeka!" is prominently displayed at the top of the image. The individuals in the picture are holding Malaysian flags and are dressed in blue shirts, which is a common color associated with the Malaysian flag. The setting seems to be outdoors, possibly in a park or a similar open area, with trees visible in the background. The overall mood of the image is festive and patriotic.


And this is the text for the cat with Malaysia flag:
QUOTE
This image depicts two cats sitting on a high structure, holding up the flag of Malaysia. The background shows a cityscape with numerous skyscrapers, suggesting that the cats are at a significant height above the ground. The scene is likely digitally manipulated to create a whimsical and imaginative setting.


This handwritten note
user posted image
QUOTE
The handwritten note on the paper reads:

```
Dear Lucinda,
Just a quick note to let you know how grateful I am for the beautiful flowers you dropped off last
```


Too bad don't have model for Ollama yet
chow1942
post Sep 17 2024, 07:35 PM

Getting Started
**
Junior Member
280 posts

Joined: Dec 2006
Using Ollama minicpm-v model for merdeka pics

Can you describe this image for me please?

QUOTE
The image depicts a group of people celebrating Malaysia's Independence Day. They are standing outdoors with several Malaysian flags in the background, creating an atmosphere filled with national pride and joy. The text "Hari Merdeka!" suggests that this moment is significant for those involved as they commemorate their country’s independence day.


can you explain and describe that image in a very detail way?

» Click to show Spoiler - click again to hide... «



TSxxboxx
post Oct 24 2024, 10:32 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


Ollama now support GGUF
https://www.tomsguide.com/ai/ollama-just-ma...ternet-required

I haven't tried it yet, but this opens up to much more choices
TSxxboxx
post Oct 26 2024, 07:25 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


chow1942 using minicpm-v can you get all the text from this image correctly?
user posted image

I only got as such using it
user posted image

But using one of the online server it got very close to complete and correct
user posted image

I wonder if my parameters is not correct or ollama/open-webui engine issue.

using llama 3.2 vision on one of the online server also give it correctly but then probably runs out of token
user posted image

I also tried on ChatGPT and everything is almost correct
user posted image
TSxxboxx
post Nov 8 2024, 11:23 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


Llama 3.2 Vision for Ollama has been released!

Trying with the earlier table, it is better than other model but still not complete

user posted image


If I crop only the table it is better

user posted image

If I crop it to be shorter table, it's better but still same split into 2 table

user posted image
TSxxboxx
post Jan 25 2025, 12:14 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


deepseek-r1 is impressively a lot better than most other model that I tested. The chain of thought makes it able to give answer that is more correct and more on point.

The default 8b I feel just a bit better than most other models, when it can't find the answer it make up something but the 14b will not. 14b also will give answer by a topic and breakdown in it as sub-topic while 8b put everything separately as an individual topic. I can only imagine how much better the 671b would be.

I ask how many e in “defenselessness”. 8b wasn't able to give the correct answer while 14b can.
user posted image

Also interesting in 14b thought on how to check to make sure the answer is correct. It also will reference back previous answer even though I'm using different model to get that answer
» Click to show Spoiler - click again to hide... «


While 8b stuck at thinking it is 11 letters
» Click to show Spoiler - click again to hide... «


Other models can't get it right
user posted image

Even ChatGPT free can't get it right in the first try
user posted image

TSxxboxx
post Jan 25 2025, 11:01 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


Asking for the numbers of letter show most of the model flaws. Gemini 1.5 also got it wrong in first try. Gemini 2.0 got it right in first try
user posted image


Even mistral-nemo can't get it right
user posted image
crazyfatt
post Apr 8 2025, 04:01 AM

Getting Started
**
Junior Member
117 posts

Joined: May 2009
From: KL,Cheras



bought rtx 5090 and wish to learn into building local ai to play. Anywhere i can get guide on doing it and which model is good now ?
chow1942
post Apr 8 2025, 07:41 AM

Getting Started
**
Junior Member
280 posts

Joined: Dec 2006
QUOTE(crazyfatt @ Apr 8 2025, 04:01 AM)
bought rtx 5090 and wish to learn into building local ai to play. Anywhere i can get guide on doing it and which model is good now ?
*
There's a few popular frameworks for running LLMs locally:

ollama - Command-line focused but easy to use
LM Studio - Very user-friendly GUI for downloading and running models
Jan.ai - Open source local AI assistant with a nice interface

For models to run locally on your RTX 5090, some good options include:
Llama - Meta's flagship model, excellent all-around performance
Mistral - Great balance of size and capability
Phi - Microsoft's smaller but efficient model
Gemma - Google's open model with good performance

This post has been edited by chow1942: Apr 8 2025, 09:54 PM
TSxxboxx
post Apr 8 2025, 11:05 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(crazyfatt @ Apr 8 2025, 04:01 AM)
bought rtx 5090 and wish to learn into building local ai to play. Anywhere i can get guide on doing it and which model is good now ?
*
For generative text, what I'm using is Ollama for server and Open WebUI for the GUI.
Download and installation follow the GitHub page
https://github.com/ollama/ollama?tab=readme-ov-file
https://github.com/open-webui/open-webui

Follow the Open WebUI instructions at the GitHub to setup and setting the web ui.

After install, go to ollama.com to choose the model that you wish to try, deepseeek-r1 is the famous now, since you got 5090, 32B model can fit into your GPU RAM. Open Command Prompt on your computer and type "ollama pull deepseek-r1:32b", after that use Open WebUI and select deepseeek-r1 model. At Open WebUI setting also there's way to download model but I feel command prompt more easier.

Some also prefer LM Studio as it's more straight forward but I never tried it before.

You should watch https://youtube.com/@azisk channel, he do a lot of generative text test.

For generative ai image, I'm using ComfyUI for the server and Flux for the model/checkpoint. There's also Automatic1111 as the server but I prefer ComfyUI as easier to do changes and apply filter or loras. Model/checkpoint is generative brain in plain word, some model good for text, some good for realistic and some for anime and many more. Go to civitai and see many different trained model for specific type of image. Some model support loras, it is to do additional changes to the model. There also inpainting, this is to mask certain part of the existing image and fill with image of your prompt. If you want to go more advance, there's controlnet. You also should install ComfyUI-Manager to manage all these extensions.

For ai image I don't have specific site for the installation walkthrough, what I done is search tutorial at YouTube and follow their steps on how to install and use those functions.
davidletterboyz
post Apr 10 2025, 11:41 AM

Der Kaiser
*******
Senior Member
4,672 posts

Joined: Jan 2003


QUOTE(crazyfatt @ Apr 8 2025, 04:01 AM)
bought rtx 5090 and wish to learn into building local ai to play. Anywhere i can get guide on doing it and which model is good now ?
*
where did you buy it and how much? Seems like the street price is way more $ than suggested price.
c2tony
post Apr 30 2025, 05:24 PM

Getting Started
**
Junior Member
80 posts

Joined: Jan 2003
From: BM, Butterworth, Penang island.


do anyone know how to close that thinking stuff on deepseek or qwen3? ollama webui of course

P/S: https://huggingface.co/jedisct1/MiMo-7B-RL-...f?download=true

can play with xiaomi ai

This post has been edited by c2tony: Apr 30 2025, 10:04 PM
TSxxboxx
post May 1 2025, 07:59 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ Apr 30 2025, 05:24 PM)
do anyone know how to close that thinking stuff on deepseek or qwen3? ollama webui of course

P/S: https://huggingface.co/jedisct1/MiMo-7B-RL-...f?download=true

can play with xiaomi ai
*
I remember using deepseek, the thinking stuff is not shown unless press the arrow beside the model name

How's the Xiaomi AI compared deepseek? Better answer?

This post has been edited by xxboxx: May 1 2025, 08:00 AM
c2tony
post May 1 2025, 01:14 PM

Getting Started
**
Junior Member
80 posts

Joined: Jan 2003
From: BM, Butterworth, Penang island.


QUOTE(xxboxx @ May 1 2025, 07:59 AM)
I remember using deepseek, the thinking stuff is not shown unless press the arrow beside the model name

How's the Xiaomi AI compared deepseek? Better answer?
*
yes, but i don't want it to show that arrow! it takes more times to show the steps, whether you click it or not

did not manage to try MiMo yet, i don't know how to load gguf
TSxxboxx
post May 1 2025, 04:29 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ May 1 2025, 01:14 PM)
yes, but i don't want it to show that arrow! it takes more times to show the steps, whether you click it or not

did not manage to try MiMo yet, i don't know how to load gguf
*
Oh you mean you don't want it to do the thinking stuff? I don't think can, those models are designed for thinking, for questions that need deep thought for answer, these kind of models are better than other models that doesn't do thinking. But if straight forward question, such as calculation then these models waste a lot of time to get the obvious answer.

Using terminal/CMD, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q8_0"
this will pull the Q8_0 8.1GB model

If you want the smaller 4.7GB model, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q4_K_M"

I tried it and the answer it given feels as good as deepseek. When feed data for it to analyze, it does take some time to process it before give the answer
c2tony
post May 1 2025, 10:11 PM

Getting Started
**
Junior Member
80 posts

Joined: Jan 2003
From: BM, Butterworth, Penang island.


QUOTE(xxboxx @ May 1 2025, 04:29 PM)
Oh you mean you don't want it to do the thinking stuff? I don't think can, those models are designed for thinking, for questions that need deep thought for answer, these kind of models are better than other models that doesn't do thinking. But if straight forward question, such as calculation then these models waste a lot of time to get the obvious answer.

Using terminal/CMD, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q8_0"
this will pull the Q8_0 8.1GB model

If you want the smaller 4.7GB model, type "ollama pull hf.co/jedisct1/MiMo-7B-RL-GGUF:Q4_K_M"

I tried it and the answer it given feels as good as deepseek. When feed data for it to analyze, it does take some time to process it before give the answer
*
thanks for the command notworthy.gif
I tried it also

Do you familiar with the thought experiment the ship of Theseus?
In the field of identify metaphysics?
If those removed planks are restored and reassembled, free of the rot, is that the ship of Theseus?

the third question it "think" about 7min

Neither is the true ship or both are the true ship?
- it's still thinking...
TSxxboxx
post May 2 2025, 08:55 AM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ May 1 2025, 10:11 PM)
thanks for the command  notworthy.gif
I tried it also

Do you familiar with the thought experiment the ship of Theseus?
In the field of identify metaphysics?
If those removed planks are restored and reassembled, free of the rot, is that the ship of Theseus?

the third question it "think" about 7min

Neither is the true ship or both are the true ship?
- it's still thinking...
*
7 Mins thinking, it really goes into deep thought sweat.gif

I asked more simpler question, "how many “e” in benzodiazepines?"
It takes almost 2 minutes to think, and the thought process is a lot
» Click to show Spoiler - click again to hide... «


On the other hand if use smaller model such as llama3.2, it is fast but give wrong answer shakehead.gif
"In the word "benzodiazepines," there are two "e"s and also one "i" but not affecting the total count of e"

Deepseek took longer than Mimo, 2 minutes plus and give correct answer
c2tony
post May 2 2025, 02:05 PM

Getting Started
**
Junior Member
80 posts

Joined: Jan 2003
From: BM, Butterworth, Penang island.


QUOTE(xxboxx @ May 2 2025, 08:55 AM)
» Click to show Spoiler - click again to hide... «
llama3.2:3b-instruct-fp16 after 2+ min answered: In the word "benzodiazepines", the letter "e" appears three times.
while smollm2:1.7b-instruct-fp16 gave me TypeError: NetworkError when attempting to fetch

XiaoMi's MiMo LLM are relative new.
Afterall they're all LLMs using the same "highway": pattern recognizing. If AI starting to understand then we might need to concern about their concise awakening laugh.gif


TSxxboxx
post May 2 2025, 10:24 PM

The mind is for having ideas, not holding them
*******
Senior Member
5,256 posts

Joined: Oct 2004
From: J@Y B33


QUOTE(c2tony @ May 2 2025, 02:05 PM)
llama3.2:3b-instruct-fp16  after 2+ min answered: In the word "benzodiazepines", the letter "e" appears three times.
while smollm2:1.7b-instruct-fp16 gave me TypeError: NetworkError when attempting to fetch

XiaoMi's MiMo LLM are relative new.
Afterall they're all LLMs using the same "highway": pattern recognizing. If AI starting to understand then we might need to concern about their concise awakening laugh.gif
*
I'm using llama3.2 2GB size, maybe that's why the answer is not good
c2tony
post May 26 2025, 08:17 PM

Getting Started
**
Junior Member
80 posts

Joined: Jan 2003
From: BM, Butterworth, Penang island.


Lately gemma3 update for 12b are annoying, they distribute it to my cpu & gpu, just won't run at 100% GPU anymore.
CODE

ollama ps
NAME                 ID              SIZE     PROCESSOR          UNTIL
gemma3:12b-it-qat    5d4fa005e7bb    12 GB    31%/69% CPU/GPU    4 minutes from now


5 Pages < 1 2 3 4 > » Top
 

Change to:
| Lo-Fi Version
0.0185sec    0.25    5 queries    GZIP Disabled
Time is now: 26th November 2025 - 07:02 AM