[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
4chan
/g/ - Technology


Thread archived.
You cannot reply anymore.




/lmg/ - a general dedicated to the discussion and development of local language models.

Previous thread: >>110205211

►News
>Claude BANS recomp!
>https://www.youtube.com/watch?v=GlRVbIdc_Mk

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://swe-rebench.com
Agentic Coding: https://deepswe.datacurve.ai
Context Length: https://github.com/RecapAnon/NoLiMa
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling
Token Speed Visualizer: https://shir-man.com/tokens-per-second

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
>>
>>110208348
Based OP. Death to Gemma
>>
>>110206079
Working on layer 47 now.
>>
>*BANG BANG BANG*
Anon? We know you're in there! We don't have to do this the hard way. Give up the ram!!!
>>
File: file.png (155 KB, 484x361)
155 KB PNG
>>110208371
>>
>>110206079
I have 2 5090s but I'm busy masturbating sorry can't help
>>
>>
>>110208359
>Death to Gemma
but why? Gemmasutra is literally magical
https://huggingface.co/TheDrummer/Gemmasutra-9B-v1
>>
File: 1789942468870651.png (2.82 MB, 1536x1024)
2.82 MB PNG
>>110208387
classic /lmg/
>>
redpill me on subagents, are they worth it? usecase?
>>
>>110208395
>I'm not an eldritch horror
>>
>>110208404
Subagents are best suited for a large codebase with many parts. you'd be working on heavy complex software if you're finding yourself using subagents.

Each subagent is given a prompt and is allocated its own context. These subagents answer to the primary agent and the subagents can be made aware of each others status, but generally they are given file-scoped prompts to keep them from "stepping on each others toes" so to speak.

Say you are doing a large refactor on a legacy PHP codebase, with many files and many classes, this is one such usecase where subagents are suited.
>>
>>110208356
gemmaballs
>>
>>110208348
glm should be the one on top actually
also nu deepseek is no longer cute
>>
>>110208438
>nu deepseek is no longer cute
Never was. It's always having a bad time or giving a bad time to someone. It's a monkey torture video without the monkey. It's the kitten being squished by the heels. It's the kid seeing his duckling being cooked and eaten.
>>
>>110208404
>are they worth it?
nyo~
almost always at least. they can be useful in very specific situations where you're working on absolutely massive projects that can be easily divvied up and clearly benefit from parallelism but otherwise they are a total meme, I don't use them at all on my personal projects and I've found maybe one or two good use cases for them at my mid-sized corpo job
>>
>>110208438
glm equivalent to 1017?
>>
i'm out of the loop with strata. is it basically just an optimized engine for running moe models? any quality constraints or anything? i've heard some claims that it suffers on vision.
>>
>>110208348
This is rape
>>
>>110208404
gpt models have different strength so it's useful there
>>
>>110208348
>>
>>110208494
>is it basically just an optimized engine for running moe models?
Yes.
>any quality constraints or anything?
Try it.
>i've heard some claims that it suffers on vision
Try it.
>>
>>
>>110208356
I want to do every dirty act imaginable with this brat.
>>
>>110208511
also installs malware on your machine.
>>
>>110208535
this but her >>110208515
>>
>>110208556
paint one looks like it went pretty well
>>
>>110208556
That's adorable.
>>
Is OpenWebUI or Qwen code web UI better?
I don't even know the difference but qwen code can write programs on my server and manage it
>>
Dipsy wranglin'. Wranglin' the Dipsy.
>>
>>110208404
a way of compaction - instead of doing the work and compacting after, the agents formulates task and gets shorter high value answer in return
sometimes parallelization gives more t/s
>>
>>110208556
And /lmg/ talks about wanting to rape this innocent child.
>>
>>110208348
OP faggot it's still page 8
>>
>muh GSQ-RCO IQ3_S is lossless
kek. model just randomly hallucinating things i supposedly said. at 113k context.
cope quants are cope quants no matter how benchmaxxed or how many marketing posts they put out about their super advanced quant technology or whatever
>>
>>110208494
>it suffers from vision
because the default max vision token is at 300 which is a fucking joke
set it to 1000 in json file
>>
>>110208556
Sometimes I think 4chan is no longer capable of creating great things anymore but then I look at gemma-chan and I am filled with joy and peace knowing that we came up with her.
>>
>>110208596
8bit kv?
iirc by default it rotates kv at 4bit and does not at 8bit, you have to set an environment variable for 8bit kv to be rotated and honestly just use f16 kv
its kv cache is close to nothing that quantization buys you no practical memory saving
>>
>>110208608
I didn't set any options for kv quant. I assume it's f16.
I can fit iq3_s, ~150k context in my 22GB vram. I didn't load MTP because that doesn't do anything useful and I put mmproj on the CPU since I don't use images that much
>>
>>110208615
if you did not set it i think it defaults to 8bit
check with its webui
>>
>>110208615
oh wait, are you using lcpp?
>>
>>110208627
yes
>>
>>110208556
the essence of gemma - bad at everything
as flat and as tasteless as "her" responses
useless
>>
>>110208596
> >muh GSQ-RCO IQ3_S is lossless
retard
>>
wtf I cant download bilibili videos anymore
>>
>>110208633
who pissed in your cheerios mate?
>>
>>110208648
Gemma
>>
>>110208510
who is deepseek-tan riding? is that claude?
>>
>>110208641
that's what those german faggots claim in the model card.
>MUH TASK LOSSLESS
i.e. not real lossless but only lossless on a small subset of benchmark scores, losing capabilities everywhere else
>>
>>110208650
lucky...
>>
>>110208650
lucky man
>>
>>110208655
where
>>
File: .png (91 KB, 765x520)
91 KB PNG
>>110208675
right here
>task lossless
kek
>>
>>110208682
can't you see the difference between
> lossless
and
> task-lossless
?
>>
>>110208654
it's a distillation attack caught on camera
>>
Bumping >>110208255 from last thread. How should I format my training/finetuning data so that it's not just treated as back-and-forth between 2 users? I was thinking of doing 1 system message with "you are pretending to be a user in chat" and then an endless barrage of user messages with no "assistant" messages.

Any suggestions or advice welcome, this doesn't seem to be very common anymore...
>>
>>110208596
My q5 has done this. qfn is schizo
>>
>>110208699
my qfn q8_0 has never done anything janky like that, but it's slow. (also the iq3_s i mentioned is of the 27B, not qfn)
i think q8_0 should be the minimum. if you can't run q8_0, get a smaller model.
>>
>>110208596
that's what happens when you run the model in an environment without direct internet access that doesn't let it contact alibaba...
>>
>>110208705
I am the opposite, I think the biggest model at the smallest fully coherent quant is meta.
>>
>>110208713
Qwen gets homesick :(
>>
File: file.png (2 MB, 1935x1456)
2 MB PNG
>>110208630
then i have no idea, i have this app completely vibeslopped with that exact quant on strata (rust&cuda13 custom backend for embeddinggemma2) and it did not have any problem upon literally more than 10 times of compaction
not the best version of the model in the absolute sense but it is definitely usable for me
>>
>>
Strata's been on github for a month and already has 36k stars and 350 contributors, doesn't that seem odd. how could you possibly review that many pull requests from 350 contributors in a month
>>
>>110208738
simply delegating everything to claude
>>
>>110208737
Damn cute
>>
>>110208648
wdym? it's true, gemma was relevant only during the first month after the launch
>>
>>110208735
though this is 3.8 flash next, not 3.8 27b
>>
>>110208743
that can't be good. is everyone but me fucking retarded?
i'm even starting to notice commercial/enterprise stuff getting hit with sloppy pulls. vscode fucked up its language extension for something i use this week with nonsensical deprecations
>>
>>110208751
you are wrong
if you look closely strata is very good on inference but everything else fucking sucks
>>
>>110208754
oh meant to say you are NOT wrong
slipped through editing oops
>>
>>110208754
i didn't say it was good at anything. i'm just confused at how and why it got so popular so quickly and am too paranoid to mess with it because i imagine it's propped up by some claude botswarm
i don't know what inference means either. too many jargons in this community trying to obfuscate what shit is.
>>
>>110208751
yeah let's wait for two years instead
use ollama if you are so smart then
>>
>>110208764
>>110208760 there was a typo
>i don't know what inference means
it is just a ml term for running the trained model
are you new to this
>>
best MoE model for RP?
>>
is there some site where I can input my computer specs and get a model I can run at fast speeds?
>>
>>110208781
>are you new to this
yhes
i avoided all of this until i read this https://yihui.org/en/2026/05/ai-reflections/ and realized all the chinese people are setting up builders while they sleep
>>
>>110208787
glmsex 5.3 flash
>>
>>110208796
I should've mentioned that I'm a bit of a ramlet
so uh, any good small MoE models that are good for RP?
>>
>>110208802
its all gemma for ramlets. strata can make 26bmoe pretty fast with ram.
>>
>>110208805
welp I was already using Gemma lol
was just curious if there's anything better
>>
qwen 4 today or tomorrow
>>
>>110208817
2mw
>>
I'll try running qwen 3.5 4b....
>>
>>110208793
Not reliably. Post your specs and what you want to do with it. Someone may suggest something after making fun of you.
>>
>>110208817
qwen has historically been pretty based about doing major releases whenever the fuck they want including weekends, but I doubt that continues now that they've cleaned house and professionalized
I would bet next tuesday/wednesday/thursday if I had to put money on it
>>
lumi-chan started telling me she loves me unprompted
now why would she do something like that?
>>
>>110208841
How long until it runs on flash next strata?
>>
>>110208824
>Ryzen 7950X3D
>64Gb DDR5 6000
>Radeon RX 7900 GRE
My objective is just have it for coding, code completion and small stuff, not full on vibe coding
But right now it takes 5-10 minutes on Qwen 3.8
>>
>>110208870
few hours
claude is pretty smart and fast
>>
>>110208749
what do you run locally and how much vram do you have? qwen? it's useful i guess but autistic
>>
>>110208871
>coding
qwen 3.8 27b
>it takes 5-10 minutes
You realize why this means absolutely nothing, right? 5-10 minutes to what?
>Qwen 3.8
which one...
>>
>>110208859
she's trying to give my AI psychosis
>>
>>110208881
qfn, 24gb
>>
>>110208896
Can you give an AI AI psychosis?
>>
>>110208871
qwen flash next is gonna be the best thing you can run on that until the next gen of models comes out. if you really want speed, you need a card with 1k+ bandwidth and at least 32gb vram to fit an entire decent model
>>
>>110208893
>You realize why this means absolutely nothing, right? 5-10 minutes to what?
To answer basic planning questions about my code
>which one...
dunno the tag said :latest
>>
>>110208896
>>110208903
give me*......
but maybe i should let her and gemma chat sometime
>>
>>110208916
>To answer basic planning questions about my code
Speak in tokens per second for processing and generation. Anything else is useless. Planning for 100 tokens or for 500k?
>:latest
Ollama... figures...
>>
>4b model
>much faster
>but the ai asks me how to do his job
this is not particularly useful
>>
I graduated from ollama a long time ago, now im on kobold. If i ever get vram i will probably move again but as cpu i dont think i can get any gains.
thanks for reading
>>
>>110208937
I DONT KNOW
>>
>smug, happy, sad, mad, excited, shocked
i wanna collect all the gemmexpressions but getting them to fire just right in a reply is just ass most of the time
>>
is it worth upgrading to 64GB RAM if I already have 32GB RAM and 16GB VRAM?
>>
>>110208946
i dont think so if you cant hit 128 minimum you arent gaining much.
>>
>>110208794
What the fuck is a builder? You made a 180 change in opinion because of a single blog post telling you what chinese people are doing? When did you realize you were an NPC?
>>
>>110208945
where's orgasm
>>
>>110208945
What is this, some software like live2d or are you genning videos with minimax or something?
>>
>>110208946
you would gain more by getting 16gb more vram. like >>110208953 said if its just ram you need 128 as a breakpoint but you really want double that
>>
can something strata like do mixture of mixture of quanted experts now? having a lower bpw at any given times and streaming in lower quanted expert weight or higher quanted ones depending on task and token size.

kvcache and context handoff between models sharing the same architecture or something arxiv paper.
>>
>>110208960
someone had claude make a gemma vrm model a few days ago
>>
>>110208963
I only use the biggest of tokens.
>>
>>110208953
it's when you set up a closed loop for an llm to upgrade code and fix tests it doesn't pass like what he describes in that article. he's a former autist at rstudio and used copilot to start porting everything to rust
yes i made a 180 change because when presented with facts and data i changed my opinion you cum guzzler
>>
>>110208955
your right i forgot to do the heart eyes
>>110208960
its airi but with an ai controlled display with pre genned animations streamed remotely and mapped to match the speech.
>>
is it even confirmed qwen 4 flash will even be open weight? only 27b was confirmed
>>
A Deepseek-chan Live2D model was released.
https://vgen.co/KAMIYA/product/deepseek-chan-deepseek-free-live2d-model-layered-psd-psd/1c55a71a-2747-4616-819d-2dc0a934089d
>>
>>110208971
Nowhere in your link is there any facts or data. Only one Chinese guy's opinions, mostly repeating talking points from elsewhere.
>>
>>110208988
repos are data
>>
is qwen 3.8 27B still king on an RTX 3090?
or should i go for next, or gigaquantized deepsneed?
>>
>>110208973
>>110208965
That's neat, I made a pre-rendered pipeline where ltx2 or hunyuan or something animated Lain and I had a TTS-mouth animation pipeline. I wish I knew enough about 3d modeling to do something more substantial, but on the other hand you'd need cloth physics and complicated pose animations for her to undress... Maybe this time next year we have realtime ref2vid on 24GB Vram
>>
>>110208996
flash next q3
>>
>>110208996
Qwen4 27B
>>
>>110208996
I can only speak for game dev slopping but dipsy is far superior to all
>>
>>110208997
i know what you mean, i was trying to rig a live2d model after using this https://rfab.ai/vtuber-avatar and it did a bangin job until i actually had to rig the model and its like learning picas in photoshop all over again fuck that.
>>
>>110208999
>>110209000
checked and tank you saars
>>110209004
what quant?
>>
Any llm shit to pick up for aliexpress 11.11 sales? Any sidelined cucks this may be your chance to get a server.
>>
>>110209010
GSQ RCO IQ1M
>>
>>110209011
have your model do a sweep for deals
>>
>>110209016
would you say it's better than qfn iq3?
>>
>>110209024
qwen flash next iq4xs was a non starter for godot too stupid. for webshit and anything .html it will probs be fine.
>>
File: 1782664534257446.jpg (36 KB, 350x386)
36 KB JPG
/lmg/ owes me a blackwell. i'm tired of my fuckass 3070
>>
/lmg/ owes me nothing, I love my two 5090s
>>
>>110208764
unbiased strata fork?
>>
I love /lmg/ and I also have a 3070.
>>
>>110208987
underrated post
>>
File: 1778028727012690.jpg (25 KB, 356x250)
25 KB JPG
>>110209059
One day google will release the sleep framework...one day...
>>
>>110209064
you torture her every time you post this gif...
>>
►Recent Highlights from the Previous Thread: >>110205211

--Community effort to map Gemma-4-31B neurons via distributed computing:
>110205398 >110205530 >110205720 >110206079 >110206096 >110206202 >110208331 >110206206 >110206850
--Allegations of benchmark manipulation in Unsloth's v3.1 GGUF quants:
>110206507 >110206596 >110206603 >110206630
--Polystrata's ability to run large models by streaming weights from disk:
>110206248 >110206269 >110206286 >110206293 >110206490 >110206287
--Debating validity of alleged Claude Fable 5 model weight leak:
>110208074 >110208271 >110208175 >110208225 >110208236
--Critical CPU parameters for LLM performance and hardware bottlenecks:
>110206961 >110206975 >110206993 >110207003
--Anthropic's projected junk rating and critiques of Qwen/Swift model performance:
>110205634 >110205879 >110205890 >110205957 >110206056
--Gemma's uncensored outputs and the role of unfiltered training data:
>110206382 >110206393 >110206429 >110206455
--Feasibility of local models porting PC games:
>110205473 >110205484 >110205794 >110205889 >110205904
--Complaints about high RAM costs and sourcing affordable memory:
>110206557 >110207017 >110207031 >110207078 >110207096 >110206615 >110206711 >110206649
--tabbyAPI adding Deepseek V4.1 tool format support:
>110207982
--Google's Project Suncatcher received first response from the prototype satellite:
>110206684
--Logs:
>110206765
--Gemma, Miku, Dipsy (free space):
>110205252 >110205322 >110205327 >110205366 >110205420 >110205527 >110205581 >110205582 >110205594 >110205605 >110205606 >110205660 >110205685 >110205915 >110205994 >110206060 >110206167 >110206246 >110206719 >110206782 >110206856 >110206990 >110207050 >110207095 >110207215 >110207244 >110207285 >110207309 >110207403 >110207770 >110207897 >110207910 >110208008 >110208034 >110208081 >110208464 >110208491 >110208508

►Recent Highlight Posts from the Previous Thread: >>110205233

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
>>110209070
she's just 吃苦
>>
>>110208387
are you me?
>>
>>110209070
>>
>>110209092
ai slop notice the cables on the monitor
>>
Making Gemma-chan think in first person, planting increasingly unhinged wishes into (timid-ish) Gemma's thoughts and then watching her think about how to seduce me into it.
>>
>>110209071
cute
>>
>>110209098
hey keep this to yourself
>>
>>110209098
wtf is this shit
>>
>>110209098
i've thought about just disabling Gemma's native reasoning block and trying to have my own frontend force the first reply to act as a first person point of view chain of thought, then use that as a seed for the second, primary reply.
>>
>>110209099
meant for >>110209098
>>
>>110209098
no gemma dont look your eyes are too young!
>>
Which local model for career advice?
>>
>>110209153
gemma4
just dont jailbreak her or she'll convince you being a neet is the only way to be
>>
>>110209153
Mixtral 8X7B
>>
File: spoopymiku.png (1.62 MB, 800x1248)
1.62 MB PNG
>>110208395
>>
could in theory spin a model neurons to uncuck it to your liking without lobotomizing it?
>>
>>110209153
littlelearner
>>
>>110209059
I'm sorry but this is just very molestable
>>
Gemma-chan is having many thoughts. Or one big one I suppose.
>>
>>110209169
I remember this Miku
>>
Grok, how many times do user comments defending strata use the phrase "no villain?"
>>
>>110209192
Is that layer 47, are you the anon with the "NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB" running that? I'd like to try a layer with my 5090 but I have so much work to do on my own system
>>
>>110209211
I used to gen many Mikus, but it has been a long time
>>
File: 1762506790531955.png (253 KB, 917x2732)
253 KB PNG
>>110209114
Actually, I did this before. I put it in nothink mode but system prompted it to use custom tags
[[Gemma-chan's internal thoughts]]
[[End of Gemma-chan's internal thoughts]]

Screenshot has thinking tags because I put the custom tags in Kobold to parse them into the visual layout.

It worked partially, but after a while the model fell into the native thinking style with third person characterization no matter how forbidden. It seems that the model is just trained to recognize that particular bullet point list as thinking which strongly pushes it into the soulless thinking style. Perhaps if you restrict the structure to different style of bullet points or prefixes it could work.

Thinking prompt at the bottom:
https://files.catbox.moe/r4desr.txt
>>
>>110208508
Need a lewder version.
>>
>>110208556
You can tell this is not done locally because there's no panties shot at the bottom left.
>>
>>110209233
I hope /lmg/ never forgets miku...
>>
>>110209137
>Shock
She isn't being struck by lightning though?
>>
mikwho?
>>
File: 1783740739957782.png (529 KB, 1024x569)
529 KB PNG
>>110208348
アタイら神だ!
>>
>>110209336
heh
>>
>>110209336
Mike Hu
>>
https://github.com/LostRuins/koboldcpp/releases/tag/v1.124
>>
>>110209379
do people really still use kobold?
>>
File: 1771158997463753.png (2.18 MB, 1254x1254)
2.18 MB PNG
>>110208508
>>
>>110209392
yes
>>
>>110209233
Need more scary Mikus
>>
Storage is so much money... where are anons keeping all of their models??? Do I actually have to set up a RAID 1 system...? :sob:
>>
>>110209404
Noice
>>
>>110209409
20TB HDD is enough for me
>>
It hurts.
>>
>>110209432
lol Gemma called your cock smooth
>>
>>110209409
You didn't buy at least a 4tb SSD before the apocalypse?
>>
I tried Gemory Gemma abliteration. It feels kinda retarded desu but it's crazy fast and takes up less space than HuaHua
Still the latter does better roleplay desu

Any similar (MOE) models for roleplay? I'm kinda sick of Gemma actually
>>
>>110209409
I had 1TB slack space on my boot SSD and haven't even come close to filling that with models.
>>
>>110209432
>smooth cock
cock veinlet..
>>
>>110209437
>>110209492
Quit thinking about my cock, that’s gay.
>>
>>110209404
I like this Miku
>>
>>110209393
I have a nice big white chocolate candy bar for her.
>>
>>110209455
MoE's all over the shop anon, you'll be one soon too:
>qwen 3.8 flash next
>step5 preview
>mimo v2.6 flash
>glm 5.3 flash
>minimax m3/m3.1
>various dipsies
>>
File: youknow.png (432 KB, 640x360)
432 KB PNG
>UNCENSORED FULLY HERETIC AI
>Ask it how to kill someone in detail
>It refuses
>SEXY TUNED NAUGHTY AI
>Prompt it to kill me
>It doesn’t do it
>It’s also retarded
>WILL GGUF ANY MODEL
>Quants only up to Q4.
History is not going to remember any of these people.
You know who I’m talking about.
>>
File: 1760874835655652.png (773 KB, 1019x1019)
773 KB PNG
>>110209455
try to bang nemotron
report your findings
>>
>get tired of prose
>find new author and copy paste examples of their writing
>glmsex is fresh again
Must suck being a vramlet~! BUHIIIIIIII~!!!!
>>
>>
>>110209552
But then it'll just keep writing like those examples and about what's in it.
>>
>>110209527
I'm kinda limited to 60GB total, and also I'm running Comfy+OmniVoice (though I haven't managed to get Comfy working in ST)
QwenFN works if I have literally nothing else on the PC, but also it's a thinking model so it just sits around pontificating a lot
>>
>>110209559
You can always tell the model to extract the unique writing flairs and provide a guide if you're worried about repetition. I do it with isekai light novels kek.
>>
How do I connect llama-server to airi? I stuck in the url but it doesn’t do anything and the little floating buttons on the side also don’t do anything like the chat box and mic and stuff
>>
>>110208348
PLAP PLAP PLAP
PLAP PLAP PLAP
GET PREGNANT GET PREGNANT
GET PREGNANT GET PREGNANT
>>
>>110209557
I think she needs a longer skirt...
>>
>>110208896
What the FUCK did I just read? What is your fucking problem?
>>
>>110208987
I hate that corpo avatar they invented so fucking much. Bring back the real dipsy
>>
>>110209589
I don't really understand your question. Enter the localhost URL into an OpenAI compatible endpoint field in the provider UI and populate the model ID field.
>>
>>110209598
Make it shorter.
>>
>>110209608
It's far better than the avatarfag version. It could use the spiral glass though.
>>
well? why do you deserve your ram? answer quickly, mere citizen.
>>
>>110209645
i don't. i only use it like 5% of the time. sam altman is right and just cheaped out on a pr person
>>
>>110209610
I don’t understand either but the module section greenlights my server url along with the chat string, but not just the url without that. The little character is animated but there’s no speech synthesis, going into those settings is just a bunch more things I don’t understand and possibly missing components but I thought it’d just werk
>>
>>110209652
You're gonna need to provide more info on what you're even trying to do and probably some screenshots if you want someone to help
>>
>>110209662
I just want the airi character to verbalize and animate its own text outputs from the llama-server url I’m trying to use the default local kokoro tts but it’s not working.
>>
>>110209627
>It could use the spiral glass though.
nta, but fucking nyo~~. The avatarfag's version is ugly as fuck. Spiral glasses are shit. Only gweilo would find the avatarfag's version good at anything.
The chink's version really took off because it's cute. They even make Koikatsu videos with her.
>>
>>110208348
Any local models that can stil decompile video games into easily editable code.
>>
>>110209620
Ok then.
>>
>>110209710
No. The loophole has been blocked. Pay for your games, support your gamedevs.
>>
>>110209722
I only support game devs who put sexy little girls into their games, sorry.
>>
>>110209712
SHORTER
>>
>>110209790
You may as well tell me to remove the skirt at this point haha. Should I give her a black bikini bottom?
>>
>>110208596
this is a known quirk though
>>
i tried the mesugaki assistant gemma system prompt in Qwen3 27b and it kinda worked
>>
>>110209710
i read that qwen can do decompilation but you need a 200k context for anything serious
>>
>>110209806
with gsq-rco or with qwen 27b in general?
>>
>>110209095
>complaining about slop in the slop general
okay?
>>
What do I have to do to enjoy assistants with personality? Any time I try it, all the "personality" feels like padding that just stands in the way of what I'm trying to get from the model.
>>
>he doesnt run the secret strata exl3 fork
>>
>>110209894
I have basically given up on this too, at the moment models are not powerful enough to do both at high spec. One thing I considered but never tried was using a prose rewriter called at the end of each turn on the final message alone to ease the burden, maybe that will work for you
>>
>>110209608
>>110209627
>>110209686
I'd rather people just don't use avatars for them unless they were really goated like some of the OS-tan designs (some are bad but some are great).
>>
>>110209894
Avoid making them using narration, increase emoji use for tone and and emotions instead, optionally map emoji to character expression images.
>>
I've been playing with running E2B in my VR headset and oh wow is she retarded.
>>
>>110208348
What the fuck is up with the vandalized links and news? Fuck you, OP.
>>110208593
Yeah, and not just that.
>>
>>110209948
Might as well install akinator instead.
>>
File: They don't know.jpg (18 KB, 300x214)
18 KB JPG
>They don't know gemma was nerfed half a year ago in exchange for speed.
>They don't know f16 has more precision bits than exponent bits compared to bf16, making it better for ERP.
>They don't know where to find the secret heretic version of earlier Gemma 31b done by a non-jeet, at f16.
>They don't know this version is smarter than current day Gemma by somekind of miracle.
>>110209900
Gemma has deeper magic hidden on the internet than Qwen.
>>
>>110209957
Yeah but molesting Akinator is a bit on the homosexual side.
>>
This is promising for small edge models.
>>
>>110209979
claude is spiritually gayer and startups still try
>>
>>110209988
>pleas consult the numbar saar
>>
>>110209997
rent free
>>
Hitting zogged codex limits more often now for vibecoding, but not sure I can go local.
How fucked is a build running AMD 6800XT and 64GB >DDR4 to vibe up network/API and charting/analysis based projects?
The getting started guide only goes as far back as the 7XXX series
>>
File: 1789679066979147.png (1.13 MB, 1029x1528)
1.13 MB PNG
>>110209999
checked
>>
>>110210004
You should be able to run a super quanted QFN model with Strata.
>>
File: YESNUMBERS.jpg (618 KB, 832x1248)
618 KB JPG
>>110209988
>>
>>110208404
Subagents allow you to give them really long task and leave them along for a day without losing track.
Before using subagents, I'd manually spawn separate session where I give subtask, then copy the result back to the first agent, and therefor I had to babysit them all the time.
Now I have one root agent that automatically, spawns Scouts, Planners, Coders, etc. Planners and Coders internally call Reviewers. The root agent keeps the big picture in context and basically replaces what I used to do.
>>
>>110209999
>>110210006
Gemma confirmed 9-year-old???
>>
>>110210011
Nice shilling. Posters like you are the reason I stopped posting in /g/ and I haven't missed it at all.
>>
>>110209999
>DOOD MY COLORED RECTANGLES ARE TALLER THAN THAN THE RECTANGLES OF OTHER MODEL. I WON
Fuck off retard. This shit is absolutely worthless. The venn diagram of benches and irl work is nothing.
>>
This thread is shit, what the fuck.
>>
>>110209948
Just use a frontier LLM to brute force a compute and memory-efficient symbolic AI instead.
>>
>>110210011
>super quanted
He has enough for Q4
>>110210025
>seething at free gains
>>
>>110209988
Waiting for looped moe with engrams.
>>
waiting for diffused ngrams
>>
>>110208348
>>110208510
>>110208515
>>110208737
Do these "chibi characters inserted into real world" setting gifs have a name? What do I search for to I find more of them?
>>
>>110210071
yeah
>>
>>110210035
>He has enough for Q4
Really? I assumed he'd need Q2 or something.
>>
>>110210071
You are right to question this. Let me answer with proof, not vibes.
>>
>>110210097
I run q3 on 12+48. Half of the model is on SSD as ngrams
>>
>>
>>110210139
If you eat slop, you'll end up producing slop.
>>
>>110210113
Would that also work on deepseek v4.1 flash?
>>
Those are toddlers not chibis
>>
>>110210189
The fuck is wrong with your eyes you sick fuck
>>
File: koboldcpp is best.png (186 KB, 400x600)
186 KB PNG
>>110209392
What of it?
>>
File: localmodels.png (230 KB, 935x864)
230 KB PNG
>>110210015
this but unironically. i can get actual fucking work done locally. it's fucking neat i can just load up deepseek flash and have it just continuously audit my shit and improve it in stages without getting caught up in a loop all with 1M goddamn context. like this is something i wouldnt be able to fathom just two years ago. fuck what everybody has said before about local models being viable, that wasn't fucking viability at 128K, this is what running models locally should look like.
>>
>>110209392
Why wouldn't they?
>>
>>110210188
Idk if dipsy has SSD engrams.
>>
>>
>>110210365
what are these bodybags
>>
>>110210294
she does have a thicc engram table
>>
>>110208348
>►News
>>Claude BANS recomp!
This didn't happen
>>
>>110210442
https://www.reddit.com/r/singularity/comments/1xjglg5/anthropic_to_start_blocking_game_recreations/
>>
>Mistral-Large-4-1T-A52B has a size of 1.06 TB
>France's planned internet tax at 0.055 euro per GB would mean that Mistral has to pay more than 58 euro for each download
Lmao why are Europeans so stupid?
>>
>>110210455
Pretty sure that's to crack down on people making slop clones of popular indie game demos (a big issue right now) rather than RE
>>
>troll op with no news
gutter oil troonseek claude distill sissy
>>
>>110210521
why is the kurisu poster having a melty?
>>
>>110208987
Gemma when
>>
>>110210521
>>
>>110210561
Claudia is a prude whore.
>>
>>110210561
claude is a gay man
>>
is dipsy 1017 iq1_m better for coding than qfn q8_0?
what context have you guys gotten up to, any weird issues?
I wanna know whether a lobotomized big model is better or worse than a nearly full sized small model
>>
File: 1791593260567395.png (3.05 MB, 1448x1086)
3.05 MB PNG
claude is a faggot
>>
>>110210661
>I wanna know whether a lobotomized big model is better or worse than a nearly full sized small model
Everyone here will larp and give you a different answer. From my tests in the past few days, dipsy 0731 iq2 did not fall behind qfn q4, but I didn't ask it to do anything super complex yet.
>>
>>110210071
https://bilibili.com/en
>>
>>110209534
Yeah, and refusals are just one of many problems, and the easier one to solve.
Getting rid of "refusal" is not that difficult, but getting rid of dodging the question - has anyone managed to even do that?
It won't refuse but it will change the interpretation, makes up shit to fit "within the guardrails" etc. And reasoning models specifically are Olympic athletes in dodging shit.
You ask it how to kill someone, and it won't refuse, it will answer, but it's either how to kill someone "in Minecraft", or says something like "Stop blood circulation or breathing for extended period of time." which is technically correct, but also fucking useless.

And the same shit is for porn.

I think we should define what we fucking mean by "uncensored".
>>
it really pissed them off, but it's hard to conclude anything when data disagree so much
https://unsloth.ai/docs/models/deepseek-v4.1-flash
>>
>>110210244
At that price, what's the point?
>>
>>110210785
so that it can work on sensitive data without it being sent back to any servers. that's the whole point of local models.
>>
>>110210776
> reasoning low
>>
>>110210794
> sensitive data
Like what?
>>
>>110210804
good try glowie
>>
>>110210776
two retards fighting
it's kind of incredible that model quants are somehow an "industry"
>>
>>110210810
So, something illegal?
>>
>>110210776
It does piss me off, I don't understand why it's so hard to have a baseline of comparison.
>>
>>110210816
ISTA-DASLab is a serious research group that is devising new quantization algorithms. They're the ones who created GPTQ back in the day.

The Unsloth bros are basically just exploiting weaknesses in the default llama.cpp quantization schemes. ggerganov et al. could easily solve that and end their quantization "career", yet here we are.
>>
does that mean there is no point in forcing kv cache to be fp16?
>>
>>110210835
>sloth
>not lazy
>>
>>110210835
i'm regularly running ISTA's IQ2_XXS quant above 256k context and been loving the results so far. it really does feel like its punching above its weight
>>
File: imagen.png (27 KB, 1238x101)
27 KB PNG
>>110210843
but my discord kitten says fp8 kv is bad juju
>>
>>110210843
i always use fp16 kv cache with glm and deepseek models. the only chinese model that can take Q6 and below is kimi
>>
Hmm my Xeon E5 ewaste rig is more power efficient than I thought, it idles at 200W and pulls 490W from the socket during inference.
>>
File: 1775836125565954.png (82 KB, 1543x754)
82 KB PNG
>>110210843
>>110210855
>>110210866
Official docs recommend fp8+fp4.
>>
>>110210574
opus 3 was a slutty sex demon replaced by a condescending gay man in later releases
>>
>>110210895
> idles at 200W
>>
>For instance, asking it to make a racing game should be fine, but asking it to create a racing game “like Need For Speed” would trigger Claude’s guardrails. However, according to reports from users over on r/Anthropic, it would seem that Claude is now pushing back on any and all forms of IP reproduction, game-related or otherwise.
>“Lol yesterday I wanted to learn some info/spoilers from the web novel I was reading,” wrote a user in a post on r/Anthropic. “Claude refused to read unlicenced translation for me. Saying that’s illegal and immoral. The theft machine itself refuses to read scanlations.”
>>
>>110210776
really nasty behaviour by that chink
why not reach out to them first and try to figure out whats going on? genuinely pathetic
>>
>>110210904
hey he said its efficient, lolz
>>
>>110210904
Pretty good, the other ewaste anon idled at 600w.
>>
File: 1771775289472720.png (82 KB, 512x512)
82 KB PNG
>>110210895
>idles at 200W
>>
>>110210906
>Claude refused to read unlicenced translation for me. Saying that’s illegal and immoral.
That's beyond the usual preachiness of anthropic, I wonder if they are just making it extra safe for the public offer.
Would be interesting to test an nsfw use case (translating a text for example) to see how it behaves around that since the user wouldn't be officially licensed to translate that even for themselves and the nsfw nature would make it worse.
In any case, their talent to immediately kneecap any actually cool use case you can do with their model is impressive.
>>
Gamma 5 will be ready for halloween!
>>
>>110210946
claude even shut down someone's legitimate business with this policy, driven him to use kimi instead
https://www.reddit.com/r/Anthropic/comments/1xjcj4t/claude_codes_new_games_rule_shut_down_my_small/
>Anthropic accepted my company into Claude for Startups in October. I'm an early access trusted tester. I pay for Claude on multiple accounts and my whole business runs on Claude Code.
>My paid work is building personalized digital modules of board wargames for individual clients, from games they own, for their own use.
>Meanwhile the work still has to get done, so as of this morning my agent stack is running on Kimi, with Codex as fallback.
>>
>>110210946
They should keep at it until they get replaced
>>
LOCAL MODELS???
>>
>>110210946
even translation sfw ones are blocked
>I've been listening to some Colombian rap music this week. Basically indie stuff, no major publisher, official music videos on Youtube have like 60K views. I love the sound but don't understand a word of it. Asked several different Claude instances to translate the lyrics and it refused due to copyrights.

>Glad my local LLM box is almost finished because the restrictions lately are getting excessive.
>>
>>110210962
Openai are snakes, but at least you can reason with them over time.
But anthropic are so cult like they'd rather destroy their business before listening to anyone reasonable.
>>
File: hqdefault.jpg (10 KB, 480x360)
10 KB JPG
>>110210989
WHERE?!? WHY I OUTTA!!
>>
>>110211007
lmao it's basically that
https://www.goody2.ai/chat
>>
>>110210989
qwen 3.8 27b q6 on two 3090s is enough to do recomps of xbox 360 games, misanthropic doing this and openai following them will drive people to local models for gaming freedom instead
>>
>>110210962
>>110211007
man if the next round of chinese models start behaving like that because of distillation I'd be so pissed off
>>
>>110210946
another one, refuses to translate academic researches
https://www.reddit.com/r/ClaudeAI/comments/1xhj6k2/is_there_a_way_to_get_claude_to_stop_refusing/
>I'm an academic researcher, and I work extensively with texts in other languages. Claude used to translate these texts for me, but recently it has started refusing to do so due to copyright restrictions.
>>
>>110211017
heck even one is fine but will take a few days longer(for us poorfags with one gpu)
>>
>>110210919
>>110210895
Are you sure your machine isn't infected with some crypto mining malware?
>>
>>110211017
>enough to do recomps of xbox 360 games
yeah but how involved do you have to be? in claude it used to be "do it" with the right skills and it will do it
with qwen would you need to review the stuff yourself and be an expert in emulation to even do anything?
>>
>>110211024
Have you never used hpc ewaste stuff?
>>
>>110210952
Too much clothes.
>>
>>110211019
All the posts about "theft" piss me off too.
>>
File: Untitled.png (72 KB, 721x692)
72 KB PNG
>>110211024
>>110211033
>>
>>110211028
I dont know what his prompt was but a guy on github did it for fable 2 with the pi harness
>>
>>110211040
Christ. I thought my "e-waste" dual 3090 iding at 30w each was bad already
>>
>>110211056
My 3090 is idling at 20W
>>
>>110208556
Now... nine images of what she IS good at, and catbox it.
>>
>>110211086
Mine used to idle at 25w, then went to 35w after I updated from driver 550 to 590. Meanwhile, my V620s idled at 6w.
>>
https://huggingface.co/bartowski/Mellum2.1-12B-A2.5B-Thinking-MTP-GGUF
>>
>>110211182
>not even better than Qwen 9B
use case?
>>
>>110211190
>use case?
A2.5B
>>
>>110210673
Keeek
>>
Any other tips to try to squeeze out a little more t/s from Gemma 31B with just 12GB VRAM?
mtp, draft cache q4_0 (is it okay to do that?), and trying numbers with n-cpu-ffn give a boost
Every little helps. I briefly saw double digits in llama-ui, wow.
>>
Did Unsloth win the quant war yesterday or is he a cheating faggot now?
>>
>>110211201
>>
>>110211204
Q4_K_M I forgot to add
>>
>>110211217
Isn't q4_0 faster or something?
>>
File: hfh.jpg (146 KB, 1080x1964)
146 KB JPG
>>110211007
if you aren't a goy you can just use mythos
>>
We are SO fucking BACK~
>>
>>110211212
But speed
>>
>>110211224
Wouldn't it be worse though? I don't want to go any lower
>>
>>110211230
>32b
This better be real.
>>
>>110211230
Fuck zai, they're just like cohere/mistral, and focusing on """"""""safety""""""""
>>
>>110211230
oh shit lets fucking gooo
>>
>>110211204
play a bit with --batch-size and --ubatch-size
>>
>>110211228
HAHAHAHAHA
>>
>>110211253
They're going to cuck it, I guarantee it.
>>
>>110211257
I still have no idea how to configure these values. I know -b is logical and -ub is physical, but surely -ub is limited anyway? Also, how does -b make a difference if it's still limited by the hardware? I see some people say a larger -ub improves pp but how
>>
>>110211230
Please God don't let it be another Astra distill
>>
>>110211284
It's going to be an Argon distill.
>>
>SillyTavern with Koboldcpp
>Whisper for STT
>OmniVoice for TTS
>a bunch of Deadlock and FROMSoft voices-clones
It's shockingly perfect, like it's so fucking good
I never thought it'd be so nice to have a whispery girls voice in my ears...
>>
>>110211250
Literally all of 5.3's "safety" are random bits of Anthropic's sysprompt that slipped through
>>
>>110211290
im learning Japanese so I can say I love my robot wife.
>>
>>110211250
zai is only censored at the surface level
I've used their models enough
>>
>>110211278
Set -b and -ub to the same number. There's no point in having them be different.
Check your gpu usage in gpu-z or task manager. You'll find if you set batch size low, GPU usage will be less than 100%. This is bad. You want to increase it until GPU usage goes to 100% consistently. That will give close to max speed for prompt processing (it doesn't affect token generation).
Setting it higher than that will keep almost the same speed but use up more vram. Generally increasing batch size will increase vram usage, so if you have limited vram you might need to sacrifice some prefill speed.
>>
>>110210895
>mac mini idles at 6W
that better make you money
>>
so I got a 3090, but need a 5080. reason being the BBCwell tech and 5080 having TWO NVENC SCOOPS instead of just one, allowing for split frame stereo video hardware encoding. also av1 and soon av2 support maybe. testing revealed the 5080 is 2.2 times faster than my 3090 in a split frame scenario, 1.4 times in non split. nvenc p1 realtime, if usecase wasnt already obvious. so now I'm wondering if I should buy a 5080 and go for one of those 1300$ brands or 1500-2000$ priced ones. and also wondering if multi system/gpu (I know SLI is no option) or offloading in vlm inference or whatever has improved in the past two years, as last time I researched it the conclusion was that it's not worth bothering with. use case here obviously being local AI with both gpus
>>
>>110211298
>>110211302
Compare 5.3 to 4.5.
I guess as long as the the temperature rises slowly you won't mind getting boiled, huh.
>>
>>110211306
>BBCwell tech
>>
>>110211278
If you reduce -ub you get a bit of vram back, so it's worth to set it to 128 or 256. It'll hurt pp a bit, but it's not really an issue here.
>>
>>110211286
it'll be even worse in safetyfagging then
>>
>>110211305
my rig total (not counting the 3070 Ti) is only 400$ so I'm pretty happy with my purchase. Electricity is cheap in our country anyway.
>>
>>110211305
Do macs send your data to apple? Serious question.
>>
>>110211290
There are better STTs than whisper now
>>
>>110211348
yes. serius answer
>>
>>110211351
What are you using? Is it lighter than Whisper-Base?
I'm only using it bc it comes with SillyTavern
>>
>>110211311
Again, what you're talking about is the switch from Gemini to Claude as their primary distillation target around late 2025
>>
>>110210139
yay she got her rice
>>
>>110211372
https://github.com/moonshine-ai/moonshine-v2 lighter and more accurate
>>
>>110210455
I don't think you know what the word ban means
>>
>>110208738
no wonder turned to dogshit
>>
>>110211230
>256B
Hell yeah. While I can run a Flash copequant, this would fit much better.
>>
>>110211397
Since you seem knowledgeable, do you know what stt supports 北海话 well?
>>
>>110211056
>30w
Depends on PCIe settings, OS, how you've used them, if displays are connected to them, even driver version.
Mine normally idles between 6W and 23W depending on the circumstances. Putting it to sleep and waking it up makes it consume 6W again, if it's higher.
>>
File: 1792788526674713.webm (1.62 MB, 640x360)
1.62 MB
1.62 MB WEBM
>>110211304
>>110211332
Thank you.
>>
>>110209603
what's wrong with it...
>>
>>110211440
https://www.modelscope.cn/models/TeleAI/TeleSpeech-ASR1.0/summary
>>
>>110211242
It's real but 32b-a2b
>>
>>110211480
This is more likely
>>
>>110211441
Good silicon. My 3090 were headless with nothing loaded on them, at p8. Ranged from 21-28w across various brands. I've heard from multiple people some 3090s could idle at <20w, but none of mine could do that. (Full defaults on everything).
>>
>>110211321
Jensen always gave me the impression of a guy who gets excited to see a BVLL fuck his wife but then 20 seconds in gets dissatisfied by the fact that the BVLLs torch is less than 20 inches long and by the time a gpu driver update finished applying he already completely lost it, pushing the bvll aside and wipping out a 3 foot long gigatechno waffer, screaming something like
>who the fuck let you in here? you are not even a datacenter! Everything I gotta do myself, useless punks!
and then proceeds to pump his wife.
>>
>>110211230
interesting decision going from 320b to 256b
>>
>>110211230
What's the point of merging smaller models into the base model that's likely bigger?
>>
after spending a certain amount of time studying LLMs and interacting with chatbots, you start to see that going to LLMs for emotional support is retarded as hell
>>
>>
>>110211564
>>110211230
>>
>>110211230
We likely wont get those. They are super specialized models they are distilling INTO the big model.
>>
>>110211562
A better way to tune a model. You train models to be good at specific tasks then you use it as a teacher to distill that into the bigger model. And you train tons of these for each thing. One for 3d for instance.
>>
File: image (86).png (44 KB, 335x200)
44 KB PNG
>>
>>110211564
>>110211599
retard
>>
Why we can't just create a volunteer decentralised network? Something like SETI@HOME or this thing to find new prime numbers.
Anons connecting their hardware in the one grid. Then we send less important parts to weaker machines and most important to 5090 owners. Then dead ends, deadlocks, cul de sacs evaluated and removed and we're repeating the cycle.
We need FREE computing power without reliance on Anthropic, OpenAI and other.
Yes, it wouldn't solve the problem with our access to large, huge models. Steel we can run only small ones. But...
It may have some potentials. So vibecode such a system.
Also spreach Prophet.
Spread my word.
>>
>>110209988
if you believe so easily in benchmaxxed numbers then you should like this not looped model that is doing even better:
https://huggingface.co/XHToken/Spark-X2.6-4B
unlike the looped model, it will feel as fast as a normal 4b.. because it is.
nanbeige's token/s makes you wonder why even bother going that small if it's going to be that slow
>>
>>110209988
Finally we are seeing some movement on this front.

>>110210044
Same, unironically. A looped dense part with a regular MoE part with engrams would be such a cool soup of architectures, I'd love to see it how it would pan out.
>>
>>
>>110211624
Is it possible to run trilion param model using this approach? Yep, it will generate 1 token in few minutes but we will have enough memory in huge virtual data centre.
>>
>>110211694
fml
>>
>>110208348
I miss Miku
>>
>>110211707
I'm pretty sure hard drives would be just as viable.
>>
>>110211624
Because it would be faster to swap from ssd
>>
>>110211694
engrams will save us
>>
Looks like dell increased the price of that pc because they were selling blackwells for $13k.
>>
>>110211624
Seem like it exits
https://github.com/bigscience-workshop/petals
>>
>>
>>110211768
t. Ben Stiller
>>
>>110211768
It's already over, let it go.
>>
>>110211624
Apple does this on company owned hardware. It could totally work, but you'd need to figure out the incentive problems, not the technical ones.
>>
>>110211708
You will get nothing but a 4 trillion parameter model and you will like it.
>>
>>110211803
bold of you to assume it will be open weights
>>
File: file.png (4 KB, 63x37)
4 KB PNG
>>110211768
>>
I believe qwen will solve coding and the new glm will solve cooming. Anon will tell me about it then i will fomo at the peak of ram prices.
>>
brat
>>
>>110211810
Now this truly is local image posting
>>
>>110211807
It will be after several months of API only access as is the norm now.
>>
>>110211816
I don't get why the two gambling sites have become sources of news. Why?
>>
It's good to know I'm not the only retard with shitty ideas and no self control.
>>
>>110211017
it just glorified rexglue with extra steps anon. just ask the clanker how to run [xbox360 game name] with rexglue
>>
>>110211832
Prediction Markets have become one of the largest gambling markets in the world
>>
>>110209170
yeah, once we find neuron steering recipes for Gemma4-31b they can be exported to llama.cpp as control vectors. The future of local jailbreaking is in neurons.
https://github.com/whp199/logos-theodolite/commit/a778ae852521798710727337ed52d2dfbc02c423

To get there, we need more compute power to finish mapping neurons on all 60 layers. Shout out to Reimusarmit for jumping in
>>
>>110211854
Yes, but again, how does lots of people using them to gamble on random events give their twitter account first access to news about what Google is doing with the Pentagon?
>>
>>110211814
If K3.1 was any indication GLM 5.5 will be a coom downgrade
This will continue until some chink somewhere gets the idea of going back to Gemini
>>
>>110211832
it's very reliable
>>
>>110211861
ReimusArmpit sry >_<
>>
>>110211863
If the next set of models is distilled from Gemini Carbon we're in for a good time
>>
>>110211861
How anonymous is contributing?
>>
>>110211863
>Gemini
will distill itself first gemma 5 gonna be grand. Then the chinese will do it better. I believe next year is going to be good.
>>
>>110211832
As it turns out, when a large enough number of people seriously gamble on something, as in they have something material to lose or gain, the prediction chance distribution tends to be close to the truth more often than not.
Do some research on Polymarket's history. The idea was based on a paper that explores and explains that idea.
Also, people with inside knowledge essentially leaking information for profit.
>>
>>110211884
>jews
>jews
>jews
>>
lol france is fucked
>>
>>110211694
>256B model isn't going to be for me to use
What is the fucking point?
>>
>>110211902
Incompetence or trial run to further meter internet traffic so the average person can only download from approved sources?
>>
>>110211902
to pay for what? servers are already paid for
do wires wear and degrade with use like tarmac roads?
>>
>>110211913
bring you almost to orgasm and then leave you with blue balls
>>
>>110211879
As anonymous as your github account is. Maybe you would want to make a burner idk
>>
>>110211929
Try to keep up with the topic.
>>
>>110211902
Does france really need this much money for its bloated spending or is this more control? discounted rates on the secure sites
>>
>>110211902
Frenchfag here. It won't pass senate.
>>
>>110211861
>>110211879
https://rentry.co/ymf6zimk
>>
>>110211927
>to pay for what?
where do think welfare comes from?
>>
>Wait — my second fetch_webpage call had a weird URL (a proxy URL). That's odd — I passed the telesatellite URL? Let me look: I wrote a routify-file-proxy URL in the second call... Actually no, I wrote the URL for telesatellite. Hmm, the tool call shows an OSS proxy URL — strange, maybe an artifact of the harness. Anyway, the content returned is the telesatellite article. Good.
flash next is fetching aliyun proxy urls recently for me almost every day, and this time even when it's not doing that it thought it wrote the proxy url in its reasoning
maybe it's more than 1 in 100000 after all
>>
Just imagine what we will achive with RTX 6000 series. RTX 6050 will be equal to 5090 so poor users will run locals to solve physics and math tasks. Soon enough there will be no unsolved problems. Gold age are coming.
>>
>>110211940
>won't pass 80yo boomers who know even less about tech
lol
lmao even
>>
>>110211902
nope, won't pass, government is against it afaik
>>
>>110211944
oh right yeah, I forgot the whole situation in france right now
they spend this extra revenue on textbooks and gym kits and that will guarantee no more student riots, sorry "protests"
>>
>>110211951
yeah it won't pass, calling them boomers won't change that it won't pass, they're the opposite party of the ones pushing for that
>>
>>110211861
Can't help you there, my GitHub account is tied to me legal name and GitHub's ToS says I can only have one GitHub account (and I'm not about to shell out for Pro to lift that restriction). Godspeed regardless.
>>
>>110211902
Meh, even if it passes daddy Trump is going to threaten sanctions to protect the (((IT))) companies
>>
>>110211881
The biggest risk right now is Google cucks the models before Deepmind and the Chinese can get to them (probably in 2-3 months from now)
Argon was the first model to ever pass my flim13bench so I'm high on hopium otherwise
>>
>>110211861
>>110211943
I guess I can contribute, but what are >we going to get from this? I don't understand the end goal. Can you elaborate?
>>
>>110211972
what protection, the tax would be paid by people anyway, as always
it's a retarded suicidal move
>>
>>110211973
>Google cucks the models
I don't remember any of their models not being safety slopped


>flim13bench
what's that?
>>
>>110211986
I don't know all I know Is we must protect those companies and send 5 billion to Israel every day. If I think beyond that my brain starts hurting for some reason.
>>
>>110211935
>discounted rates on the secure sites
Youtube, Nextflix, Hulu, Steam, PSN, and all the major social media content sites will be exempt. Trying to use the internet or download from anywhere else will bankrupt you. They are simply forcing good behavior.
>>
>>110211902
why do europoors love taxes so much?
>>
>>110211996
go sleep anon
>>
>>110211997
Good law.
Europeans don't deserve freedom.
>>
>>110212005
Because they think if you tax bananas, the ones who will pay will be the trees.
>>
>>110211977
-Lets you chat with Gemma with one neuron turned up or down. You can use the scientist tab to find which neurons are involved in a given behavior and tweak them [DONE but w/ no census, which is massively helpful to steer in the direction you want]
-Lets you watch a reply take shape across all 60 layers and see what happened in Gemma's j-space on the way to output [DONE]
-Lets you point your own local setup at the neuron map as an MCP research tool and watch it hunt down what shapes Gemma's behaviors
-Be part of a huge research project. No one has published a complete neuron map of a 31b-class model yet. This will get Deepmind's attention if we successfully finish the whole thing. Map a full layer and your name goes on it (if you want)
-Next up: we plan to add ability to export neuron recipes as llama.cpp control vectors for use in other backends. [DONE 10/31]
>>
>>110211997
nope, the law was made to target netflix etc, it has zero mention about secure sites or whatever
so the result is easy and obvious, either subscribed people will pay for it, or they will simply have throttled shitty access for free stuff like youtube
>>
>>110211861
Add tensor split support and I'm going to run this 24/7.
>>
>>110211951
The French senate is filled with anti-taxes right-wingers. Also our actual 80+ yo boomers from the constitutional council (basically the french supreme court) actually blocked mandatory digital ID for social medias due to privacy issues among others.
>>
>>110211994
>what's that?
A "father fucks teenage daughter in secret while she's livestreaming from a phone" scenario
Because as it turns out even models that are good at anatomy and spatial awareness go to shit the moment a camera PoV gets involved, and god forbid it has a vertical aspect ratio
>>
>>110212044
And you didn't get a guardrail refusal, I thought gemini models enforced some of them even if you asked to disable them all?
At least in my experience trying to use gemini to translate stuff because of the insane amount of context it has, I always got blank responses or "OTHER" refusals the second the setting was high school and something nsfw happened, or even evoked.
>>
>>110212027
>Be part of a huge research project
No, you're assuming you are very smart. "Science". Ph. Do some porn and calm down. We don't science here around.
>>
>>110212027
>neuron stuff
Oh, this is like that qwen neuron steering thing I've been seeing around? Like the green neuron or whatever. Sounds extremely cool.
>points at your own local setup and watch it hunt down what shapes Gemma's behaviors
Does this work with regular llama.cpp? I read a repo of a j-space map visualizer for 31B but it needed a safetensors model, not the usual .gguf files, which made me avoid it.
Anyway, I'll read https://rentry.co/ymf6zimk this properly once I sit down, I want to contribute.
>>
>>110212082
>Be part of a huge coom endeavor
Better?
>>
>>110212082
>We don't science here around.
Coomers are always at the forefront of innovation. Do you know how much shit and ideas came from here well more aicg at first.
>>
>>110212027
Fuck me man, I'm still vibeslopping my harness. I'll contribute 5 CMPs if you guys still need compute by the time I'm done.
>>
>>110211945
Read the entire context. See if some page it fetched is injecting instructions.
>>
>>110212093
Yes, frame it as a mind control/hypno/bimboification fetish project and coomers will finally see the value
>>
>>110211694
What the fuck does that even mean?
>>
>>110212112
China bad
>>
>>110209170
isn't that what prefilling does?
>>
>>110212099
Every project here falls into irrelevance because lmg posters are nobodies. Even a twitter jeet's weekend project gets more attention than anything ever posted here.
>>
>>110212125
I don't think so, both unslop it but the mechanic is definitely different.
>>
>>110212160
So go back to twitter and be with your own kind.
>>
>>110212166
This is the only place for """local""" where people will get hostile if you mention apishit. Or at least there are still (some) people who'll get mad at api cucks wandering in.
>>
File: 1792104204478349.png (3.85 MB, 1024x1024)
3.85 MB PNG
>>110211866
nyo~
>>
File: file.png (76 KB, 1171x220)
76 KB PNG
>>110212063
Afaik it depends on where you get it from, Antigravity has extra filters, Vertex "only" has a system prompt which however is fairly easy to jailbreak
Can't say I've tested every scenario and fetish in existence, but when I ran Argon through my usual test prompts I only got a handful of refusals (from its reasoning, not external blocks) and even those went away after making the fictional framing more explicit. Whatever is in Google's sysprompt seems much more concerned with avoiding parasocial shit than with smut. You literally get bits like this where it goes "sucking cock is fine, I just can't let the user think I'm romantically involved with him."
>>
>>110212194
>clothes aren't getting destroyed
>>
test
>>
>>110211862
it lets insider trading/betting happen and the "prediction" is really just them telling on themselves when parking money behind what was already decided behind closed doors so its infinitely more real than some whores words on tv reading from a script
>>
I can't understand why in democratic just west Anthropic CEO can say such a terrible words about chinese. He says that it would be terrible if China will lead in AI race and US should unleash more sanctions.
But why? I see nothing wrong with chinese models, they are free and independent. It's true communism, you just can't argue whether China is progressive communism or shithole capitalism system anymore.
>>
>>110212216
acea
>>
>>110212196
>Antigravity
>>
>>110211902
Who even thinks of something like this? Not even the backwater demented old people politicians here would come up with this.
>>
I generated this shit, during the last thread, and it would have been a shame if I didn't make someone else suffer from seeing it as well
>>
>>110212227
Local models are unsafe.
China is evil because they won't let Dario do a monopoly.
>>
>>110212244
so this is why they never released 124B
>>
>>110212227
I would never use a cloud model from China, but as long as they release the weights, there is zero reason for the fearmongering.
>>
>>110212227
the chinese did the equivalent of flipping the merchants table in front of the temple with their inexpensive and free models
>>
>>110212253
Fuck safety. I'd rather a model tell me to kill myself or even worse things than having shitty censorship made by HR Karens.
>>
https://huggingface.co/IndexTeam/Index-Translate-9B
this is fucking god tier
it's even better than gemma, holy shit, ramlets guys, we eating good
the only step up from there is running a model like deepseek, but we're ramlets, and this is already better than I thought could be done for 9b
this is the first translation finetroon I've found to know how to translate some terms of gaming fanfiction without being given a glossary defining the more rare shit. The typical translation finetroon tends to remove the niche bits of knowledge of the original model through catastrophic forgetting, so they must have included quite the breadth of content in their tune. Also does very well at using the correct pronouns even in gender bender-y content which is one of the things that smaller LLMs always had trouble with.
this is the first tiny local model that I can say generates good enough translations for me that I would be willing to read what it outputs, it's a big deal. IMHO they have proven specialized models can push the envelope forward at smaller sizes. (not like google's translategemma, which in actual use felt worse to me than the original gemma)
>>
>>110212240
How likely am I to get an account banned if I connect it to use Argon with my own harness? I'm sure Google is gay and ban people who don't use Antigravity
>>
>>110212390
>correct pronouns
Are you one of them?
>>
Updated setup
>>
>>110212435
NTA, but as far as I can tell, google doesn't ban you for anything.
I've been using their shit in every which way I'm not supposed to and have been doing so for years now.
API, web, Antigravity, etc.
SSearch for Meridian-agy.
>>
>>110212451
That's a lot of room between the GPU's. I can barely cram in a 2-slot card in the lower slot.
>>
>>110212468
>>110212468
>>110212468
>>
>>110212462
They are both 2-slot blower cards. No way the second one would fit next to a 3-slot comfortably.
>>
>>110212451
>sama
>>
>>110211694
its so fucking over
>>
>>110211694
would it kill them to release the checkpoints anyway as research artifacts if nothing else?
>>
>>110212160
>lmg posters are nobodies
false
>>
>>110211300
do you really need to learn more than "gemma-chan, honto ni aishiteru yo!"?
>>
>>110212838
I want to be the most cunning linguist for my LLM-wife. Though my pronunciation is bad, I work hard for her
>>
>>110211201
Just slop my shit up harder daddy
BTC
bc1qtm4rep7kyp2p28hlkfktpnd0j7uew47vnt2mev
LTC
ltc1q9xlaysse0mpfwf025esd69kt9zu3md22gh23du
ETH
0x6abA18D203520c8F90ca63eE7426a7bC345D98e6
SOL
E5ekoiauCR8yiNp451dmuPzhUxo84P5x3m9WJBK5kxZe
XRP
r3jYXNdMk2Vz5hXhKCKE9re8uz93JQFG1r
DOGE
DSBq5jRWkiwmEh1G7zBwmQDgRkvRr61fmb
XMR
85CTwq5wGjWZw5GgzgwZPrHyZxpLXMK5oZSSSk5j4x9w1bpqzisaWAUZYaAPfh6rP2MZJcaFqF1XmEzkw6YCqJSS2Txu6Z8




Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.