How usable or sensible is this hardware/software to run an LLM (e.g. LLAMA with voice assistant? I have the idea to use it for Assist in HomeAssistant (so I don't need the openAI API, which can get expensive if used often)
Yes, hardware is one thing, but Google's software support is lacking. I had my own struggles with it. NVIDIA CUDA is supported much better. Plus, with Google, you never know when they'll kill it off and leave you high and dry. I can't recommend it.
I ended up going with Libre 3 flavors the top one fully blows away Raspberry 4 when pricing gets even close to $100 for every item it’s like auto eject as hobo happy hobby researching…
Good thing you have in your hand think 2~3 years ago went to buy a older Coral w/ priced $60 nxt thing every direction scooped up & vendors were selling’ like @ $190 even $300 even if one could afford it was they were hard find a/ lead time was spaced like months for the next available batch {same thing w/ the older Versions of Raspberries
Sorry but ist way to complicated as to be usefull. It is way easyer to talk to a gpu as to this thing. Also why i need all this stuff? This makes no sense to me. Install Dirvers and up to do not that nonsense. In oder to be usefull for the people this musst be was simpler other wise no one will use it and this will end up as waste Technologie. I am a programmer not a systemintegrator. Give me a language and a api and a driver to work with but dealing with Background software and mqtt cals is not usefull and i hate this aproche. If you wanna brake your system go a head do this nonsense but i defently wont write any code for this as long it is not idiot prove installed and used.
It's 2024 and Linux still makes people use a dos prompt for things like a caveman. Hey look I'm dialing up WOPR with my 2400 baud modem in my terminal program.
At 4:06, the special file under /dev will only appear when you connect the device. This command does not check for the driver module. The appropriate commamd would be something like modinfo
I appreciate the Raspberry shade being tongue in cheek. But its getting a little overly salespitchy at this point with the Zima Blade ;). Ive dipped my toes outside of the Raspberry Pi ecosystem plenty of times, but its still the king for its ecosystem and support community. All this niche hardware has its place and im glad theres competition…. But once you start caring about more than what the Raspberry Pi can offer, you really should be looking into actual industrial production grade equipment. Its really not as cool as you make it out to be, to be the prettiest girl, with no friends, at the…. Special Ed school dance. Harsh and offensive… I know. But it gets the point across very well about how i see the Zima Blade
Trying to be on the cuttingt edge of the hobbiest ARM community is just silly, and just means you have outgrown it. 3 months later and all my Pi 5s are handily eauipied with m.2 and PoE, and its given appropriate time for the 3D print community and 3rd party accessory providers to provide complimentary supportive equipment.
Messing with systems like these is all about learning, socializing, communicating and having fun. Raspberry Pi has that on lock.
iMHO once you've outgrown the Pi you should move right on over to Jetson and higher end Arduino setups, and start investing in real industrial quality setups with userbases to match. But that's just me. Invest in platforms that have legs and big companies and communities begind them makes spending in this space an actual investment, and not just expensive adult toys.
If the Pi 5 supported faster USB and pci-e it would defeat the entire point of the community it targets and affordability it maintains even with the shortages making prices go up on the Pi chips themselves. Having anything faster then Gen3 pci-e in a project like these would be like putting a Ferrari engine in your lawnmower, and even with the "affordability" of the Zima, would put you right back into insane price points. With a small fraction of the comminty and support you would have when stuff inevitably is buggy or poorly documented.
And. . any time something in Raspberry Pi land doesn't work…. Its your opportunity to figure it out and get it fixed… And actually have your efforts appreciated by a huge number of people
How usable or sensible is this hardware/software to run an LLM (e.g. LLAMA with voice assistant?
I have the idea to use it for Assist in HomeAssistant (so I don't need the openAI API, which can get expensive if used often)
If we could run multiple AI on a single system, we could absolutely have a functional android very soon.
Anyone calling this easy is insane. Disliked.
How do you know for sure this will be completely local and untapped tho
Yes, hardware is one thing, but Google's software support is lacking. I had my own struggles with it. NVIDIA CUDA is supported much better. Plus, with Google, you never know when they'll kill it off and leave you high and dry. I can't recommend it.
Hi, are you for hire, as I need help with building a prototype?
Wonder how much faster coral sped up the id over just cpu
ZOMG not you using docker for the frigate server and not containing the MQTT in the same compose…. zomg that just made me want to defenestrate myself.
What is this awful audio? Dude you gotta remove the compressor and any other filters. This is so grating on the ears
WHY DO YOU TEAR YOUR VOICE. AHHHHHHH
This guy is like a super gay version of the raspberry pie guy
can you run llama-3-7b via oobabooga webui on this setup?
Will that coral tpu speed-up my work on steam deck my roboflow image detection is getting 6fps?
I was checking on this but this is already deprecated
I ended up going with Libre 3 flavors the top one fully blows
away Raspberry 4 when pricing gets even close to $100 for every item it’s like auto eject as hobo happy hobby researching…
Good thing you have in your hand think 2~3 years ago went to buy a older Coral w/ priced $60 nxt thing every direction scooped up & vendors were selling’ like @ $190 even $300 even if one could afford it was they were hard find a/ lead time was spaced like months for the next available batch {same thing w/ the older Versions of Raspberries
which screen display, mouse and keyboard are you using for the raspberry pi in this video?
Use some of that AI to fix your audio
Wondering if the s capable of ruining small llms such as phi 3 4k instruct withabdffeeebt sbc of course maybe the 16gb ram. What s u thank?
Sorry but ist way to complicated as to be usefull. It is way easyer to talk to a gpu as to this thing. Also why i need all this stuff? This makes no sense to me. Install Dirvers and up to do not that nonsense. In oder to be usefull for the people this musst be was simpler other wise no one will use it and this will end up as waste Technologie. I am a programmer not a systemintegrator. Give me a language and a api and a driver to work with but dealing with Background software and mqtt cals is not usefull and i hate this aproche. If you wanna brake your system go a head do this nonsense but i defently wont write any code for this as long it is not idiot prove installed and used.
can you use that tpu on pc to train deep learning much faster?
Didn't we already have it with Intel neural stick 2
I got the Mini PCIe version of the Coral TPU
I put it in a Xeon E3 1265 V2 Dell Optiplex 7010 usff
It stomps man
It's 2024 and Linux still makes people use a dos prompt for things like a caveman. Hey look I'm dialing up WOPR with my 2400 baud modem in my terminal program.
At 4:06, the special file under /dev will only appear when you connect the device. This command does not check for the driver module. The appropriate commamd would be something like modinfo
Who cares about vision. We want llm on pi
I appreciate the Raspberry shade being tongue in cheek. But its getting a little overly salespitchy at this point with the Zima Blade ;). Ive dipped my toes outside of the Raspberry Pi ecosystem plenty of times, but its still the king for its ecosystem and support community. All this niche hardware has its place and im glad theres competition…. But once you start caring about more than what the Raspberry Pi can offer, you really should be looking into actual industrial production grade equipment. Its really not as cool as you make it out to be, to be the prettiest girl, with no friends, at the…. Special Ed school dance. Harsh and offensive… I know. But it gets the point across very well about how i see the Zima Blade
Trying to be on the cuttingt edge of the hobbiest ARM community is just silly, and just means you have outgrown it. 3 months later and all my Pi 5s are handily eauipied with m.2 and PoE, and its given appropriate time for the 3D print community and 3rd party accessory providers to provide complimentary supportive equipment.
Messing with systems like these is all about learning, socializing, communicating and having fun. Raspberry Pi has that on lock.
iMHO once you've outgrown the Pi you should move right on over to Jetson and higher end Arduino setups, and start investing in real industrial quality setups with userbases to match. But that's just me. Invest in platforms that have legs and big companies and communities begind them makes spending in this space an actual investment, and not just expensive adult toys.
If the Pi 5 supported faster USB and pci-e it would defeat the entire point of the community it targets and affordability it maintains even with the shortages making prices go up on the Pi chips themselves. Having anything faster then Gen3 pci-e in a project like these would be like putting a Ferrari engine in your lawnmower, and even with the "affordability" of the Zima, would put you right back into insane price points. With a small fraction of the comminty and support you would have when stuff inevitably is buggy or poorly documented.
And. . any time something in Raspberry Pi land doesn't work…. Its your opportunity to figure it out and get it fixed… And actually have your efforts appreciated by a huge number of people
too rapid jumping around to follow and limited ability to follow commands used. seems very rushed
Thank you for this. 😉 Have a zimaboard 832, with coral tpu USB & PCI