This post has been edited by ipohps3: Jan 28 2025, 01:22 PM

moooore memory...

slow la. better just use proper graphic card instead.

wonder if it is using the NPU, GPU or CPU in the M4 Mac Mini. if NPU, no need dedicated GPU also can. also, inside the M4 Mac Mini got GPU too. main thing is memory.

i have 3 gpu on floor

can i setup r1 ?

most likely GPU. they will need extra work to get NPU doing the hard lifting. also NPU is optimized for performance/watt, it wont be faster than GPU with AI accelerator.

Who need that kind of setup??
Graphic designers?

Sampah punya AI pun ada hype.

dis 1 is localised, no internet

i download ady the android version

not bad the chain of tot n reasoning all listed 4 u to see

can register new account?

also only possible to run without internet if open-source.

Note that the open source model you can run the distilled version, not the full one they hosted in cloud. You need crazy amount of fast GPUs to make the full version workable. Have fun enjoy heating you room.

This post has been edited by kingkingyyk: Jan 28 2025, 02:22 PM

better just pay chatgpt plus. only usd20 per month. lol

I am running a RM50million business company with just my M1 MacBook Air.

what's the difference, preciousss?

What does it do? What's the difference between running the one online?

If you already have some big VRAM GPUs, open source LLM lets you utilize the computing power at no cost (well, you could argue there is TNB bill).

Bruh. The full model is also open source:
https://github.com/deepseek-ai/DeepSeek-R1

ollama too:
https://ollama.com/library/deepseek-r1

Edit:
lol you edited

This post has been edited by hellothere131495: Jan 28 2025, 02:27 PM

Obviously you don't know AI at all

If DeepSeek is sampah

All AI related tech giants won't cirit-birit in a row

running the 1.5bn q4_k_m locally on my NAS
slow af, but it worked so I dont have to work with o1 free-limit

Same. The same model. Just that you probably won't have the chance to run the online one. It's full 32 bit and 671B parameters. What you can run is the distilled (and quantized) version, like the qwen and llama3.1 8b one that has been distilled to respond like the original deepseekr1

16GB VRAM sufficient to run o1 level intelligence ?

ok.....so world peace achieved?

1.5b 4bit is slow? what spec are you using? btw, you probably still need the o1. 1.5b 4 bit is a shit model that hallucinates a lot.

The speed of China Tech is really terrifying

Luckily such power is in the hand of highly civilised nation

China is the future of humanity

https://github.com/deepseek-ai/DeepSeek-R1/...s/benchmark.jpg
Here is the comparison. The full 671b models need over 400GB of memory to run, which is out of reach for most people.

Distillation transfers the knowledge to smaller model (i.e. feeding the QA chain), but smaller model has way fewer parameters so they won't be generating result so well.

The list of distilled models:
- DeepSeek-R1-Distill-Qwen-1.5B
- DeepSeek-R1-Distill-Qwen-7B
- DeepSeek-R1-Distill-Llama-8B
- DeepSeek-R1-Distill-Qwen-14B
- DeepSeek-R1-Distill-Qwen-32B
- DeepSeek-R1-Distill-Llama-70B

32B can let you run on RTX5090 nicely with context (the QA chain is involved to generate further response), but how many of us can justify buying that 5 digits GPU + heavy TNB bill just to run this?

This post has been edited by kingkingyyk: Jan 28 2025, 02:33 PM

At this rate

Full fledged offline AI robot is possible

ah lol. 16GB vram is a kid toy that can only run 4-bit quantized small models.

o1 is a big model. you probably need around 400GB vram to run it (in 4 bit probably). To run the full 32 bit idk need how much. lazy to calculate.

No. a war has just started? see the news? china deepseek is getting attacked, probably by US

https://ollama.com/
Try the 12b/14b models yourself.
The general performance is quite acceptable, but due to the amount of parameters, won't work well in depth.
7900XTX is the best price-performance card for such purpose. Gives you 24GB VRAM and just cost you 5K.

N100 processor je, no GPU, its truenas machine
originally installed ollama just for fun
now considering putting it on my desktop

8 M4 units and no world piss? Best it can do is run Crysis on max?

So it's better than Nvidia?

I think it is still a long way to go before we can get AI to the same level as shown in A.I. movie by Steven Speilberg (2001). What we have now shows the promise of what could be achieved. But the power drain is way too much to put it into a machine that not just needs to run the AI, but still needs portable power to move around as well. There are way more things that need to be improved before we can even get a robot similar to the one that is shown in the Interstellar (2014) movie.

Then we have other benchmark to reach, which seem still very far out of reach.

Robot in Lost in Space (1960s TV series version, not the new movie or Netflix version)
R2D2
C3PO

etc etc

Hackers are doing everyone a favor by doing free blackhat test on new AI system. So that it can be improved better in the future.

Aiya at home you need robot vacuum, automated home system, not this bull crap to ask hallo how is the weather tomorrow

Nah man he's building Skynet at home

Trust me bro

Can someone explain this in a easier way?

https://medium.com/@artturi-jalli/how-to-in...ep-65245873fea8

can calculate toto supreme winning combo

Can run on ipong?

arguably, this is like back during mining trend? people buy hundreds and thousands of GPUs just to mine the hell out of it plus the insane electric bill monthly

Genuine question, cant do crossfire / sli with 2 gpu to save cost instead?

Don't "worry"

It won't be too long

AI itself will solve the bottleneck

Just like how AlphaFold (DeepMind) solved 50-year-old protein folding problem in hours

This post has been edited by GagalLand: Jan 28 2025, 04:12 PM

The whole thing?

Remember Ironman 1? Tony Stark (Deepseek) just built a workable arc reactor (AI model) in a CAVE. And much smaller (way cheaper in DeepSeek case).

First Tesla come in, then BYD(deepseek) come in?

full 671b iinm is slightly below 1400GB memory required /gg

NVlink can, like using nvlink on two rtx3090, you get 48GB shared vram.

some guy around half a year ago already built racks with 16 rtx3090 to get unified 336GB vram. But I dunno how he set it up with what operating system etc.

But it can also mean some smaller companies can setup own server using local run deepseek r1 system for own employees for sensitive data liao

This post has been edited by terradrive: Jan 28 2025, 05:14 PM

rtx3090 last to nvlink, newer consumer dont have, must buy those fancy card

but can it run crysis?

Just wait for the China version for AI clustering. Sure cheap like cabbage prices.

No wonder China having unemployment issues, all taken over by AI already, they just never announce publicly.

Like many other Chinese AI models - Baidu's Ernie or Doubao by ByteDance - DeepSeek is trained evades politically sensitive questions.
When the BBC asked the app what happened at Tiananmen Square on 4 June 1989, DeepSeek did not give any details about the massacre, a taboo topic in China.
It replied: "I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses."

Tim cook does 80% of his work on an iPad, and he runs a trillion dollar company.

Cool

This post has been edited by likefunyouare: Jan 28 2025, 06:01 PM

By the time you and I won't be witness liao

aaaaaa imagine if i can stuff vram into gamelady 2b and ask it act out aaaaaaaaaaaaaaaaa i gonna coooommm!!!

That is for big companies More GPU reduces training time hence you can get a big model in more feasible time or have more turnaround time for tuning.

You can, but you need to consider the packages too. MSDT only gives you 2 x8 PCI-E slots.

me against him. he confirm win lor. but I am sure he used latest and greatest iPad.

I'm running local DS R1 14b on ASUS 1070 GPU

maybe that is using app version, since it routes the question back to deepseek servers in china

Why would someone want to do this instead of subscribe to their services?

Ok I have locally run deepseek R1 32b Q4 now, this is the response:

There is censor filter in cloud deployment. regardless of vendor.

Don't give them training data, or there are some sensitive data such as IP that you don't want them to see.

This post has been edited by kingkingyyk: Jan 28 2025, 07:06 PM

i got this in 8b

I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses.

Spoken like a true bukit person

so meaning
ironman arc reactor = deepseek
nuclear reactor = chatgpt?

lower parameter probably more dumb lol

Just matter of time everyone will throw their dataset into the new method.

This post has been edited by kingkingyyk: Jan 28 2025, 07:19 PM

My pc already crapping up with flux nf4, and will fully locked up after several gens lol

Time to throw some more money and make tnb happy

political topics might not be that accurate since it is affected by the training data

for example it is fairly light on the details on gamergate while showing more bias on "toxicity and harassment in gaming culture" which is untrue. toxicity is minor issue on gamergate, I can pull up the real event on inner circle on people preserving the evidences of gamergate that was always being deleted by the woke ppl (it's more about journalism corruption)

This post has been edited by terradrive: Jan 28 2025, 07:46 PM

i'm genuinely hope for it. who the heck can afford nvidia h100, and thousands of them again.

you probably asking something unethical. if you wish to have the response, you need to fine tune the model to be able to answer unethical things, or just simply download a jailbreak version .gguf file from somewhere else.

Wow. The 春晚 got robot group dance. Amazing.

So U guys mean with different 8b, 16b, 32b or 64b all are giving different responses??

for end users probably not worth it.. for enterprise.. they pay only fraction of what it cost if running on chatgpt

of course. the smaller the model, the more it's prone to making mistake, hallucinate, and not following instructions properly

Then to make a comparison with chatgpt, what model would be appropriate to compete?

Main use of AI:
NSFW stories lol

the 671b, full bit unquantized model.

hey. any freelancer willing to do a bit coding for hire?

im planning to build a chatbot using deepseek for my online futsal court booking platform. API ready.
deepseek api cost is very attractive compare to chatgpt.

im a bit lazy to code on the side now.

different topic:
US tech stock dipped after deepseek blew up. but I think its a knee jerk reaction.

deepseek is open sourced. chatgpt and the likes can make reference and make changes to their codes. and with their massive AI chip that is only available to them.
its gonna come back MASSIVE.

its good for business and consumers. I think massive discount gonna come soon to chatgpt and the likes when are on par in term of cost with deepseek.

I think have a long position on US AI stocks is still a good look


thoughts?

This post has been edited by syyang85: Jan 28 2025, 11:16 PM

Its free right? Then what point of using the smaller version compared to 671b?

The smaller version are less demanding in terms of hardware (VRAM in particular)

cheap and good

full bit 671b requires around 1400GB memory size of either the system RAM (if running cpu AI) or GPU VRAM.

Not rich enough to have that monster pc

This post has been edited by terradrive: Jan 29 2025, 06:30 AM

No need, just run small model with tool support will do.

Use less resource, but much weaker performance.

Ask deepseek to code?

Maybe wait a couple months will have deepseek r2 since liangwenfeng just met with Chinese Premier and gotten Hangzhou govt support after rocking US's AI industry? Imagine having entire huawei cloud for his team to train the next version of deepseek.

But with full bit, what is the performance compare to chatgpt? Same or even higher than chatgpt?

Haven't gotten over Deepseek's R1? Try Deepseek's AI image generator, Janus Pro 7B



This post has been edited by Penamer: Jan 29 2025, 10:14 AM

the competitor for o1

but of course doesn't have image generation and video generation capability under this deepseek r1.

So after all? Still chatgpt win ke?

Sorry. Missed the bus.. apa guna AI setup at home? Can tell me when the collapse of crypto?

if you wanna pay $200 dollars a month for unlimited o1 use lol

plus the ability to run deepseek r1 offline solves privacy issues which is very important for some users

I hope the electricity bill is worth it

I am wondering with my house desktop setup, what version of deepseek can it use.

It depends on memory size. I can only answer for GPU part. If you have 24GB VRAM gpus like rtx3090, rtx4090, you can go for quantized 4 bit deepseek-r1:32b. If you have 12 or 16GB gpu then you can try 14b. 8b for 8GB gpus. But this is for the distilled version quantized 4 bit, not the full bit. full bit uses way more memory.

It can also be run on CPU only, I didn't read much about that but I saw ppl running 70b using 9950X with 64GB system memory but the response is slow. Another built a amd dual epyc system with 768GB memory that managed to run the full bit 671b with 6-8 tokens/s with total system cost at his place around 6000usd.

Damn my GPU old let ... Rtx 3060 super I think... Can do what? Just subscribe chatgpt?

gpu requirements
https://apxml.com/posts/system-requirements-deepseek-models

If you used to use the free version of ChatGPT, this won't beat it. It is already has way more parameters than what home user can (afford to) run.

Run 14b models and see which one works best for you. That's it. At this model size, there is nothing that can know everything.