Is SE [going to be] selling our content for AI model training? And what exactly does "reinvest back into our communities" mean?

_{«You» is hereafter used interchangeably to refer to Prashanth Chandrasekar, Stack Exchange, and the company’s employees. It does not refer to the creator of the question.}

If the intention really is to bring the value back to the community, by charging the companies using our content for their AI models, and spending that money on the platform, I whole-heartedly agree with the intention.

I am also happy that you seek to address the license violations that these companies are making. It's appreciable that you enforce this. At the same time, selling our content under terms acceptable for any buyers, does not sound compatible with the existing license. You have previously illegally relicensed our content. This is not acceptable. You can however provide an opt-in ability for all users, for them to agree to have their content sold under a different license, on the basis that the money earned goes back to the community.

Some users may want to reserve themselves against their content being used to train these models. Will you provide an option for this?

If the money is not given back to the community, and instead directly spent at your other commercial products, or sent straight into the pockets of the investors, I will whole-heartedly hate you for it. The community will feel used. This is the content provided by the community that you are selling, not your product. I will be extremely uncomfortable with anyone capitalizing on this content, in this manner.

internet forum for computer programming help

It's not clear to me if this is a statement by the CEO himself, or a grave mislabeling by Wired. However, Wired already labelled SO "The programmer Q&A site", so a clarification about this mislabeling will be most helpful.

This quote caught my eye as I was reading:

When AI companies sell their models to customers, they "are unable to attribute each and every one of the community members whose questions and answers were used to train the model, thereby breaching the Creative Commons license," Chandrasekar says.

I look at it from a different angle than you, however. When I asked ChatGPT:

Were you trained on Stack Overflow data?

It replied:

As an AI language model, I was trained on a large and diverse corpus of text, which includes various sources of data such as books, articles, and web pages. However, I cannot confirm whether Stack Overflow data was included in my training corpus or to what extent it may have influenced my training.

OpenAI, the creators of the GPT-3.5 architecture that I am based on, has not released detailed information on the exact sources and amount of data used to train their language models, including mine. However, it's worth noting that my training data was pre-processed and filtered to ensure that it meets ethical and legal standards, such as removing personally identifiable information and sensitive content.

Regardless of my training data, my responses are generated based on statistical patterns and relationships between words and concepts that I learned during training, and I do not have any direct access to external data sources.

I did regenerate this response once to get what I was looking for. It's probably a mistake to read any nuance into these answers. Given that ChatGPT can code, it's virtually impossible that Stack Overflow data wasn't used. Due to the nature of the process, the model can't determine which bits of text it used to produce any specific output.

Of course I can't either. At one point I learned how to use the tar command, but I couldn't attribute all of the input I used to generate my answers. (Honestly, I don't recall writing one of these answers in the first place!) Charging other companies for using Stack Overflow data feels like a violation of the spirit of the endeavor. I mean, if it's not illegal to sell a book based on the data, I don't suppose it's illegal to sell the data itself. It just . . . why would anyone buy it unless the free copy stops existing?

Coincidentally, I ran across a site that uses the data dump from the dearly departed (in my opinion) Stack Overflow Documentation project. There's no attribution or even a link to the data. It's pretty annoying since the whole point of CC BY-SA is to maintain a link between the creator and consumer of content.

In my opinion, companies that use Stack Exchange content to train their language models should be very clear that they are using it. If Stack Overflow wants to charge for using its brand or special access to the data, that seems fine to me. In some ways I'd like to avoid blame for the trivial influence my answers have had on ChatGTP's buggy and sophomoric output. As long as my (possibly buggy and sophomoric) output remains freely available to anyone looking for it, I'm good.

Update after thinking a bit more

I asked Google Bard my "Were you trained on Stack Overflow data?" question based on Laurel's comment:

Yes, I was trained on a massive dataset of text and code, including Stack Overflow data. This data includes questions, answers, and code snippets from Stack Overflow, which is a question-and-answer website for programmers. This data has helped me to learn about a wide range of programming topics, and to be able to answer your questions in a comprehensive and informative way.

That's a more direct acknowledgement. Then I asked "How much wood could a woodchuck chuck?" (as you do) and got this answer from Bard:

A woodchuck could chuck about 700 pounds of wood on a good day, with the wind at his back. This is based on the research of Richard Thomas, a New York fish and wildlife technician who calculated the volume of dirt in a typical 25–30-foot (7.6–9.1 m) long woodchuck burrow and had determined that if the woodchuck had moved an equivalent volume of wood, it could move "about 700 pounds (320 kg) on a good day, with the wind at his back".

Sources

1. www.myth-weavers.com/showthread.php?t=395451
2. github.com/kevin-hartman/MIDS
3. en.wikipedia.org/wiki/How_much_wood_would_a_wood

This tells me two things:

1. Woodchucks are active little buggers.
2. It's possible for a large language model to cite sources.

That means Stack Exchange Inc could ask for Stack Exchange/Overflow content to be listed as a source. This would immediately make answers from language models more useful. It would also directly address the attribution concern. Whether or not they come to a commercial agreement is an entirely separate question.

Based on my experience as a former Community Manager, it seems likely management didn't investigate how this announcement would go down when the community found out about it. As a community manager at College Confidential I learn about potential revenue sources weeks or even months before deals are signed. I just got out of a weekly team meeting where we discussed upcoming clients to make sure that legal, product, marketing and community are ready the moment the contracts are signed. I'm not sure why this isn't part of Stack Overflow (the company) culture, but keeping community managers in the loop really makes a difference for our entire company.

From my perspective, the community benefits when the company that hosts it stays viable as a business. Meanwhile a company that hosts a community benefits when it correctly values that community. Practically that means a community-centered business might need to say no to profitable deals that harm the community. Fortunately, Stack Exchange has an excellent community team who might help the business side of the company generate revenue without eroding the value of the community. As always, I wonder if they will listen.

I seriously don't know where to start.

SE has always emphasized the CC license model which includes attribution and you have to be a lawyer to figure out that TOS change introduced in 2018 meant content is licensed to SE under additional licensing terms which gave them right to use our content and do whatever they want to do with it, including removing attribution. See: Do Stack Exchange’s ToS mean that the user-generated content is double-licensed to them?

Now, even if reading the TOS made it clear that this change means dual licensing, I wouldn't immediately call that a problem because there are certain situations under which SE may need those additional terms in order to run the sites. For instance, when a user decides to delete their account, or the account gets deleted because of rule violation, or similar.

There is nothing wrong with that.

Also, SE is a for-profit organization, and there is also nothing wrong with SE making money on top of user-provided content. But, again, I have never envisioned in my wildest dreams that making money off our content could include selling that content in a way that would remove attribution.

Training AI on our content to improve site capabilities, like better search and finding duplicates and similar questions or even for improving writing question titles is one thing, but training AI on our content in ways that will allow AI to generate full-fledged answers and other stand-alone information where attribution will be lost is a completely different story.

I didn't sign up to have my content used for generative AI training that will remove attribution.

First, I'd like to say that the intent of what Prashanth is saying is very simple: to return value to the community for the work that you have put in. The money that we raise from charging these huge companies that have billions of dollars on their balance sheet will be used for projects that directly benefit the community.

The stance SE and (or) CEO has seems to be: "Companies are using SE content for generative AI, violating the CC license, so let's make some money out of it."

Thanks, but no thanks!

I would expect that SE with their position can at least try bringing up the attribution topic in public discourse and media, and pursue higher standards for generative AI, where AI would have to include attributions with generated texts.

I know that this may not be enough to prevent abuse of our content by companies that train AI models, but it is the minimal first step.

And last but not least, how did the community find out about those plans?

Did CEO Prashanth Chandrasekar come here personally and discuss with the community before making such plans, or at least had he the decency to inform the community first hand about those plans? Did he come here and explain how will community benefit form this money?

No, he told about those plans to other media and wrote an extremely vague post on the Stack Overflow blog that didn't even mention anything close to selling our content for training AI.

CEO is as detached from the community, as he can possibly be.

This kind of behavior on behalf of the company and the CEO can only lead to another wave of valuable members leaving the sites and reducing their contributions.

And eventually, when the quality of content drops (and the sites are already full of trash), the results of this lucrative sales deal will be garbage in, garbage out.

In the end this simply means that SE will sell the community-created content under a non-CC commercial license that has no attribution requirement, circumventing the usual restrictions that apply to all the content here.

Of course we don't know the details, but I see no other conclusion that fits the statements by the CEO. Though this is rather unsettled legal territory, so I think it is quite possible that some future court cases might obsolete all ideas SE had here.

I'm not entirely sure SE has the legal right to do this, but the ToS are a bit too hard to interpret for me as a non-lawyer in this case. I would read the statements by the CEO as an indicator that at least SE thinks they have the legal right to do this.

I'm not entirely opposed to this idea from the start, but I am a bit wary as this is essentially selling content that was created entirely by the community for free. The part that pushes my opinion here deeply into the negative area is the way SE has handled this so far. Reading this from a random article instead of having it announced here is a bad start, even some SE employees did not know about this. Reading this from a random article just after the CEO posted a blog post on the same topic of AI that was devoid of any substance feels deliberate, and is rather insulting. Explaining to us now that SE is doing this to protect our right to attribution by adding themselves as a middleman that just charges for content they didn't create is an outright insult.

Another danger here is that SE might be tempted to add some kind of restrictions to ensure that they can charge big companies for our data. I think that would be a grave mistake and I would strongly urge SE to be extremely careful. If they stop publishing the data dumps or restrict the API in ways that interfere with regular users, it will harm relations with the community sigificantly.

Right now, as far as I understand it the legal doctrine that purveyors of large AI projects have been operating under in the US is the idea that their use is Fair Use under US copyright law. Fair Use covers cases where you do not need permission of the creator/rightsholder to use the content.

However, this has not really been tested in courts yet. See some articles about this:

https://creativecommons.org/2023/02/17/fair-use-training-generative-ai/

https://www.linkedin.com/pulse/fine-line-fair-use-ai-copyright-concerns-geoctrl/

https://www.theverge.com/23444685/generative-ai-copyright-infringement-legal-fair-use-training-data

Importantly, if training an AI is Fair Use, the license for content you post here doesn't matter. The license is for giving people permission to use your copyright-protected content under certain terms. Fair use is not protected by copyright, so it doesn't matter what terms you set.

If SE wants to sell access to the data here, that would suggest they believe Fair Use will not hold up for this use of content, and presumably that they're willing to defend that idea legally, otherwise they'd be selling something that's already free. If you like the license on your content here and are bothered by companies stepping around it to train their AI models, that's probably a good thing for you, unless you feel comfortable taking on legal action against the large companies using this data yourself.

At that point, the license terms do apply, so it's worth considering whether it's possible to comply with those terms while selling the data for training. That seems tricky to me and I don't know enough to comment on it.

tl;dr: It seems there are a lot of legal issues to work out here, and importantly, the terms of the CC-BY license are not where it makes the most sense to start.

I have a similar hope to Andreas that profits from our contributions go back into improving the platform.

In particular, I'd like to see design, discussion, and development time get put into the long lists of bug tickets and feature requests.

• MSE: is:q [feature-request] or [bug] -[status-completed] -[status-by-design] -[status-declined]

• MSO: is:q [feature-request] or [bug] -[status-completed] -[status-by-design] -[status-declined]

• That's not even counting the same search queries performed on all other network sites' meta sites.

As for how much of that profit, the part of me that doesn't want to expect too much says at least 50%, and the part of me that sees a company possibly about to make a bunch of money off the backs of people who have always been on a mission to freely make the internet a better place says as close to 100% as you can reasonably make it.

Update: Community Asks Sprint. Very exciting. Hopefully this initiative stays alive long term.