Zos_Kia

That’s a room temp take at best

But then how am I supposed to use your “research” to make imaginary claims on generational attention spans ?

“i have collected some soil samples from the mesolithic age near the Amazon basin which have high sulfur and phosphorus content compared to my other samples. What factors could contribute to this distribution?”

Haha yeah the top execs were tripping balls if they thought some off-the-shelf product would be able to answer this kind of expert questions. That’s like trying to replace an expert craftsman with a 3D printer.

What kind of use-cases was it, where you didn’t find suitable local models to work with ? I’ve found that general “chatbot” things are hit and miss but more domain-constrained tasks (such as extracting structured entities from unstructured text) are pretty reliable even on smaller models. I’m not counting my chickens yet as my dataset is still somewhat small but preliminary testing has been very promising in that regard.

Most projects I’ve been in contact with are very aware of that fact. That’s why telemetry is so big right now. Everybody is building datasets in the hopes of fine tuning smaller, cheaper models once they have enough good quality data.

It’s especially frustrating as the whole point of the Google search page was that it was designed to get you out on your way as fast as possible. The concept was so mind blowing at the time and now they’re just like nevermind let’s default to shitty

This comment shows you have no idea of what is going on. Have fun in your little bubble, son.

If I understand these things correctly, the context window only affects how much text the model can “keep in mind” at any one time. It should not affect task performance outside of this factor.

Yeh, i did some looking up in the meantime and indeed you’re gonna have a context size issue. That’s why it’s only summarizing the last few thousand characters of the text, that’s the size of its attention.

There are some models fine-tuned to 8K tokens context window, some even to 16K like this Mistral brew. If you have a GPU with 8G of VRAM you should be able to run it, using one of the quantized versions (Q4 or Q5 should be fine). Summarizing should still be reasonably good.

If 16k isn’t enough for you then that’s probably not something you can perform locally. However you can still run a larger model privately in the cloud. Hugging face for example allows you to rent GPUs by the minute and run inference on them, it should just net you a few dollars. As far as i know this approach should still be compatible with Open WebUI.

There are not that many use cases where fine tuning a local model will yield significantly better task performance.

My advice would be to choose a model with a large context window and just throw in the prompt the whole text you want summarized (which is basically what a rag would do anyway).

some myths are hard to kill honestly

I mean it is also true for crypto. BTC, the most energy-hungry blockchain, is estimated to burn ~150TWh/year, compared to a global consumption of 180 000TWh/y.

Now is that consumption useless ? Yes, it is completely wasted. But it is a drop in the bucket. One shouldn’t underestimate the astounding energy consumption of legacy industries - as a whole the tech industry is estimated to represent just a few percents of the global energy budget.

To clarify: AI is NOT a major driver of CO2 emissions. The most pessimistic estimations place it at a fraction of a percent of global energy consumption by 2030.

You’d be surprised! We already had banks, insurances, newspapers and other kinds of information businesses. They did employ a huge lot of secretaries.

No the article is badly worded. Earlier models already have reasoning skills with some rudimentary CoT, but they leaned more heavily into it for this model.

My guess is they didn’t train it on the 10 trillion words corpus (which is expensive and has diminishing returns) but rather a heavily curated RLHF dataset.

• it’s not actually AI
• it’s just fancy auto complete/ glorified Markov chains
• it can’t reason it’s just a pLagIaRisM MaChiNe

Now if I want to win the annoying Lemmy bingo I just need to shill extra hard for more restrictive copyright law!

Reasoning has nothing to do with knowledge though.

You should have asked chatgpt to explain the comment to you cause that’s not what they say

Yeah it always strikes me how religious extremism is framed. You rarely hear about christian extremists, who operate in the open on all social networks.

Yet, you could argue that Christian extremists have done more harm to western societies in the last 20 years than any Islamic group.

That’s a nice hypothetical but the facts of this case are much simpler. Would you agree that a country is sovereign, and entitled to write its own laws? Would you agree that a company has to abide by a country’s laws if it wants to operate there? Even an American company? Even if it is owned by a billionaire celebrity?