Martins Udris PRO

Thanks for the input but these happened all when temp = 0.0

My guess is, since I use mostly datasets generated from voice, the models are one thing when they are talking like a human in day to day life, but completely opposite when they are feeling like a scientist, producing a long text..

Thats actually interesting. As Hararri has stated "Information connects two points" and human interaction mostly doesn't convey facts like textbook does, it merely just plays role of social status building and probing other human beings, so if model is trained on casual human speech, one might come up with idea that its trained on noise. And from that perspective its irrelevant whats more nutritious in "water cooler talk", perhaps its better to switch around to see "who is with whom" - one day its "Yolk", next day "White".

Either way models are predictors that are optimized according to training data. Noise in - noise out. In SFT surely You can nudge model to generate predictions that are more grounded and stable, meaning it will ground them in previous knowledge, not just generate text that sounds right.

In fact i have finetuned recently corpus pretrained interesting raw model, look up my datasets and work. https://huggingface.co/martinsu/tildeopen-30b-mu-instruct

Since its first token groups(Yolk|White), that sets whole story(in Your example), obviously problem is too high temperature, if one desires more deterministic outcome - set it lower. And since those (Yolk|White) groups are predicted somewhere at start, it doesn't matter for how long the predicted tokens go(how long is output). Next tokens already attend to them and thus generate some "reasoning" that enforce that, as it was instruction-trained to do.

Is there a possibility to add RAG via simple SERP in space? I think ATM most integrators think in terms: how this model plays with RAG-like tasks. How well it focuses and works with context window populated with grounded knowledge. Can it detect what docs are off and should be ignored? Google SERP with returning not only hits, but snippets of text(like short docs) are good addition. SERP is really cheap and blazing fast.

Ultimately, we will see various "flavors" of the same large models, each reflecting distinct "worldviews"—much like Google Maps displays different country names and borders based on a user’s geolocation. This approach will serve as a legally compliant solution to regional regulatory requirements. Currently, such adaptations are implemented crudely—often through rigid guardrails that suppress sensitive topics or force outputs toward regionally "approved" responses. In the future, however, MLOps systems could dynamically select the appropriate model variant for each user, mirroring Google Maps’ long-standing practice of geo-localized content delivery.

But this is not only about real stuff, say, model is trained to generate predictions that are more "rational", it would make it suboptimal fantasy LARP generator, even with excessive prompting.

But if MLOps team is stuck in its options of model choice and must wrestle the model to adhere to something model resists, oh that's a bad situation.