Students with access to ChatGPT produced higher-scoring marketing recommendations in a randomized study at Bocconi University. Training in causal reasoning helped participants offer ideas that differed more from their classmates' suggestions.
The experiment involved 1,053 first-year economics, finance and management students. They were divided into four groups: ChatGPT Edu with GPT-4o, causal-reasoning training, both, or neither. Each group proposed ways to increase awareness and use of Bocconi merchandise among alumni.
Bocconi reports that ChatGPT access increased scores by about 0.86 points on a scale from one to five, against an estimated control-group score of 2.09. Submissions contained more ideas and more closely resembled expert recommendations.
The reasoning exercise encouraged students to explain causes and effects and propose more varied ideas. Those differences did not earn higher marks under the predefined marketing criteria. Students continued applying the reasoning training when they also had access to ChatGPT.
The university described the Bocconi and OpenAI collaboration in its August 27 report. The findings concern this assignment with GPT-4o, not a general measure of students' learning or the performance of newer models.