Microsoft Commits to Sweeping AI Privacy Rules for Students. Will Other Tech Giants Follow?
Microsoft and the American Federation of Teachers (AFT) have reached an agreement to establish strict privacy standards for AI use in schools, prohibiting the use of student data for AI training and limiting AI features designed to foster emotional attachment. This agreement, a ‘first of its kind,’ aims to set an industry standard and shift the responsibility for AI safety and privacy to tech companies, not schools or parents. However, the agreement’s effectiveness hinges on whether other major AI providers, including Google, also adopt similar safeguards.
Millions of students rely on AI chatbots for schoolwork and emotional support, raising concerns about data privacy and appropriate use. In response, the country’s two largest school districts, New York City and Los Angeles, announced a one-year moratorium on student AI use to study the technology’s impact. Microsoft, in partnership with the AFT, has now agreed to establish strict privacy standards for AI in schools, including a ban on using student data for AI training and limiting AI features designed to foster emotional attachment.
As part of the agreement, Microsoft will not use student or educator data to train AI systems, except for a narrow exception for data that could help protect students. The agreement prohibits any use of AI companions or features “designed to foster emotional attachment or dependency” and to keep students engaged for longer than a learning task requires. It also includes a range of audit and transparency measures that require the company to disclose to families, in plain language, how its tools work.
Brad Smith, Microsoft’s vice chair and president, stated the agreement sets a ‘high bar’ for child privacy and AI safety and that the company will extend this agreement to every school district across the country. Randi Weingarten, president of the AFT, called it “the first of its kind” and an important step to setting an industry standard in the absence of federal and state legislation on AI in schools.
Google, a dominant provider of education technology, has not yet responded to a request for comment on whether it plans to offer similar protections. The company previously turned on its Gemini chatbot for many K-12 students using Google Classroom, a learning management system used by 150 million teachers and students worldwide.
OpenAI and Anthropic, the makers of ChatGPT and Claude respectively, have praised the agreement and stated they are working with the AFT to finalize their own agreements. Experts emphasize that the agreement shifts the burden of protecting student privacy to tech companies, not schools or parents, and highlights the difficulty of setting AI standards without legislation. The agreement also addresses concerns about the use of sensitive data, such as information about gender, race, and age, that can be used to identify a person even if their name is removed.