Tried the 7B distilled model on open webui ollama
not better than phi3

perhaps to try on text2sql later

Yes it's free and open source. The point of using smaller version is because 671b needs a very powerful machine to run it. You need a football field size data center, lots of Nvidia H100 GPUs cluster to power the model. It's impossible for normal people to run it, even you buy the best PC you still can't run it.

So, for us, we can only run the smaller models like 8b, 14b, or 32b.

comparable to chatgpt o1. Even slightly better than chatgpt a bit.

If you're an AI user, then just use the chatgpt or the deepseek website. The downloaded 4 bit smaller models ain't going to perform better than the chatgpt website or deepseek website, which are running full bit model, with giant parameters.

Those 4 bit models are meant for production use like when you want to build a chatbot and sell to company. You won't ask the company to subscribe to chatgpt right? You need to provide them a full standalone working chatbot.

Or, when you are working with sensitive data, then you need to download open source models and use it privately.

Or, you are AI specialist. You do research in AI. You download the models to study them.

Or, you are simply just a hobbyist or enthusiast in Natural Language Processing and just want to have fun with the state of the art technical things in AI.

Just used their app, I would say it's pretty good for a free version so thumbs up for me 👍

Orang sampah tau tak DeepSeek. Obviously is a gimmick distrub share market.

DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit

https://ca.finance.yahoo.com/news/deepseeks...-144534774.html

This post has been edited by James8899: Jan 30 2025, 10:24 AM

Gimmick?

Kesian

Uneducated but arrogant ppl are really sampah

Orang sampah macam tak peduli.

My mac mini can only run deepseek r1 14B model. not bad juga.

I wonder what level of deepseek I can run with 2xxeon6133 + 512Gb ram + 2xRTX4090.