It would requires some cards but technically all is done.
backend is:
ollama run deepseek-r1:671b
and the front end can be chosen from a lot of MIT ollama interfaces.
The above command runs on 8 Γ 80 GB A100 and works quite well.
Ofc you need to scale for usage. I got some experience in that.https://x.com/PascalBauerDE/status/1883672122356380145
Please authenticate to join the conversation.
Open
eu/acc
About 1 year ago
Get notified by email when there are changes.
Open
eu/acc
About 1 year ago
Get notified by email when there are changes.