Dave explains the reasons why and the steps needed to set up your own local AI engine ala ChatGPT. For my book on the autism spectrum, check out:
Helpful links and tips:
Install ollama:
curl -fsSL | sh
ollama serve
WSL Kernel Update Package:
Run the UI Docker Container:
Run UI with Docker:
docker run -d -p 3000:8080 –gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data –name open-webui –restart always ghcr.io/open-webui/open-webui:ollama
[ad_2]
source
Wouldn't it be easier/better to take a machine and just run Linux as a default OS without using a virtual machine and sharing resources with your main OS?
im sorry Dave, Im afraid I cant do that.
You asked about a pigeon. But what would be better, would be a laden swallow, but then you would need to specify African or European.
Dave love your channel. Keep it up
Did you say 512GB of ram ? 😲
The birth of thousands of baby Skynets.
‘Do you want to play a game?’
Michael
Awesome
Read the license terms for any "free" AI models. They own everything you input and everything it outputs. Forever. It's like the origin story for Facebook. Zuckerberg just wanted chick's numbers. All your data are belong to us.
Haha, using windows only to install Linux.
5:01 – “…maybe it’s an ASD thing like watching the washing machine…” 😂🤣😭😂🤣 I feel so seen right now! 🤣❤️
Thank you very much for the video. FWIW, your final incantation to launch the web UI didn't work on my system – something to do with packages installed with snap not being able to see the graphics card drivers (?) Following the installation instructions on Open Web UI's page seemed to do the trick though.
Well, toss aside the speed/hardware/cost issue. That's something you can grow into or even Moore's-Law your way into (i.e. be patient).
The real issues are: dialog & scripting with persistence that's even forkable, so that you can maintain contexts just like people maintain repositories and even do multi-party engagement with it. That's especially the case, if you're going to put up a public-side interface to this.
That's a primitive form of knowledge-base integration, which gets to the next item: real-time updating and learning, not just training by some batch process at "update time". That's not trivial, and it's an issue that is independent of the scaling issue alluded to in the first sentence: how to integrate short-term memory into the long-term memory that is the model, itself. Trainability is also an important issue, yes, but I'd be more concerned with the ability to interface with components for a hybrid architecture that includes a logic and math engine and knowledge-base engine.
The "advantages" that the greater resources put into the major AI-providers' models diminish exponentially because of the neural scaling law, so you can go a long way to getting into the same ballpark as them, without the blow-up up in resources that they have or used to get there. Hybridization could blow through that wall, slingshotting right past the big players, if it's done right – in a race to get there before they do. A model of your own is good, but you really need hooks into these other things to go with it, or you're just cosplaying OpenAI in the minor leagues. I want to move this to a more modular form, actually, as curriculum training; and also to mold a personality type. An already-provided pre-trained model is just a starting point to launch this off of, but only if the extra hooks are integrated into its design.
"I'm sorry Dave, I can't do that…"
In the WSL2/Ubuntu set up, I can get Ollama running and I added Docker via command line. I used Apt not snap. The Open WebUI localhost:3000 UI comes up but it’s not connected to the Ollama process. It seems like there is some kind of Windows networking issue? Has anyone seen something similar and solved the issue?
Thank you, sir.
Hey Dave, as an former employee of Digital Equipment Corp for over ten years, I love the t-shirt.
Do you know how I could put one behind a pay wall. At the moment when you give someone the link they have it. I want to set it up so I have control
At first, from the title, I thought I’d set one up. But $50k machine…🔥💥⚡️🫥
U may want to inform you audience that each request to AI needs at least 500 ml of H2O and amazing amount of energy
Nice info
Didn’t understand a word but it was interesting
What a cool rabbit hole… I installed it on my unraid server and loaded 3.1 and i'm hooked. I have no idea how it works and am like a kid in a candy store. Surprisingly this was the best video to get me up and running. I'm already thinking about a heavy lift system build because If my P2000 does this good, I can't wait to see what it can do with some amped up hardware.
Thanks. I will give this a shot on an Nvidia Orin
"Run it at home" on a $35k computer.
hi Dave
will a rtx3060 12gig card work ?
I've got a wickedly deep library of ancient texts that I'd really like to utilize as my own data set as well a many modern texts that are otherwise banned and censored out by the Judenpresse AI clownshow.
When a legit AI module is trained up on the truth that these criminals so desperately want hidden forever, it's checkmate… Especially, if that information gets an audience.
Dave, you're always producing high quality entertaining and educational content. Thank you for such dedication and pushing through to share such rich information with all of us.
This is possible, but you'd need to run llama 405B to come even close to GPT4o, let alone o1 or o1 preview. My system can only run the 8B one, so slightly better than GPT3.5? This is heavily outdated already. In addition, it's free to use even o1 mini, so why would I want to run llama locally and torture my PC? But that's me, I'm sure there are people who'd want to experiment with this, I just think performance hit is not worth it. Thanks for the video.
Which are more objective models? In my brief testing the llama model is very left leaning and "defensive" of google, I'm not based in the US so couldn't care less but it's bias is very obvious and annoying for research purposes.
Truly hoping and positive, I've seen from other "Youtube content creators" 3 of these, each was not accurate. Before I do it, can I get some comments verifying this?
Total cost =? Did I hear it cost more than $30k?
What's the point ?
So it has to be Linux? I saw on the Ollama website that there is a windows installation. Does it work?