16GB VRAM sufficient to run o1 level intelligence ?

ok.....so world peace achieved?

1.5b 4bit is slow? what spec are you using? btw, you probably still need the o1. 1.5b 4 bit is a shit model that hallucinates a lot.

The speed of China Tech is really terrifying

Luckily such power is in the hand of highly civilised nation

China is the future of humanity

https://github.com/deepseek-ai/DeepSeek-R1/...s/benchmark.jpg
Here is the comparison. The full 671b models need over 400GB of memory to run, which is out of reach for most people.

Distillation transfers the knowledge to smaller model (i.e. feeding the QA chain), but smaller model has way fewer parameters so they won't be generating result so well.

The list of distilled models:
- DeepSeek-R1-Distill-Qwen-1.5B
- DeepSeek-R1-Distill-Qwen-7B
- DeepSeek-R1-Distill-Llama-8B
- DeepSeek-R1-Distill-Qwen-14B
- DeepSeek-R1-Distill-Qwen-32B
- DeepSeek-R1-Distill-Llama-70B

32B can let you run on RTX5090 nicely with context (the QA chain is involved to generate further response), but how many of us can justify buying that 5 digits GPU + heavy TNB bill just to run this?

This post has been edited by kingkingyyk: Jan 28 2025, 02:33 PM

At this rate

Full fledged offline AI robot is possible

ah lol. 16GB vram is a kid toy that can only run 4-bit quantized small models.

o1 is a big model. you probably need around 400GB vram to run it (in 4 bit probably). To run the full 32 bit idk need how much. lazy to calculate.

No. a war has just started? see the news? china deepseek is getting attacked, probably by US

https://ollama.com/
Try the 12b/14b models yourself.
The general performance is quite acceptable, but due to the amount of parameters, won't work well in depth.
7900XTX is the best price-performance card for such purpose. Gives you 24GB VRAM and just cost you 5K.

N100 processor je, no GPU, its truenas machine
originally installed ollama just for fun
now considering putting it on my desktop

8 M4 units and no world piss? Best it can do is run Crysis on max?

So it's better than Nvidia?

I think it is still a long way to go before we can get AI to the same level as shown in A.I. movie by Steven Speilberg (2001). What we have now shows the promise of what could be achieved. But the power drain is way too much to put it into a machine that not just needs to run the AI, but still needs portable power to move around as well. There are way more things that need to be improved before we can even get a robot similar to the one that is shown in the Interstellar (2014) movie.

Then we have other benchmark to reach, which seem still very far out of reach.

Robot in Lost in Space (1960s TV series version, not the new movie or Netflix version)
R2D2
C3PO

etc etc

Hackers are doing everyone a favor by doing free blackhat test on new AI system. So that it can be improved better in the future.

Aiya at home you need robot vacuum, automated home system, not this bull crap to ask hallo how is the weather tomorrow

Nah man he's building Skynet at home

Trust me bro

Can someone explain this in a easier way?

https://medium.com/@artturi-jalli/how-to-in...ep-65245873fea8

can calculate toto supreme winning combo

Can run on ipong?