Just a guy shilling for gun ownership, tech privacy, and trans rights.

I’m open for chats on mastodon https://hachyderm.io/

my blog: thinkstoomuch.net

My email: [email protected]

Always looking for penpals!

  • 6 Posts
  • 97 Comments
Joined 1 year ago
cake
Cake day: December 21st, 2023

help-circle

  • For simply productivity like Copilot or Text Gen like ChatGPT.

    It absolutely is doable on a local GPU.

    Source: I do it.

    Sure I can’t do auto running simulations to find new drugs and protein sequencing or whatever. But it helps me code. It helps me digest software manuals. That’s honestly all I want

    Also, massive compute projects for the @home project are good?

    Local LLMs runs fine on a 5 year old GPU, a 3060 12 gig. I am getting performance on par with cloud ran models. I’m upgrading to a 5060ti just because I wanted to play with image Gen.




  • nagaram@startrek.websitetoLefty Memes@lemmy.dbzer0.comOpen Source washing
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    1
    ·
    edit-2
    10 days ago

    Which is funny since that does solve a lot of the problems.

    If it’s completely open source at least.

    Like OS data sets and model that can be ran locally means it’s not trained on stolen data and it’s not spying on people for more data.

    And if it runs locally on a GPU, it’s no worse for the environment than gaming. Really the big problem with the data center compute is the infrastructure of getting that data around.




  • I am a fan of LLMs and what they can do, and as such have a server specifically for running AI models. However, I’ve been reading “Atlas of AI” by Kate Crawford and you’re right. So much of the data that they’re trained on is inherently harmful or was taken without consent. Even in the more ethical data sets it’s probably not great considering the sheer quantity of data needed to make even a simple LLM.

    I still like using it for simple code generation (this is just a hobby to me so Vibe coding isn’t a problem in my scenario) and corporate tone policing. And I tell people non stop that it’s worthless outside of these use cases and maybe as a search engine, but I recommend Wikipedia as a better start almost Everytime.




















OSZAR »