The practice of medicine relies on accurate diagnosis. However, the diagnosis of many medical conditions involves assessments that invite varying degrees of subjectivity. External and arbitrary factors can influence physicians’ diagnostic assessments in conditions ranging from heart attacks to neurodevelopmental conditions like attention deficit hyperactivity disorder (ADHD). Quantifying this subjectivity is challenging, however, particularly for neurodevelopmental and psychiatric conditions where subjective assessments of behavior are common and important. Halloween, a holiday characterized by excitement among children, could present a natural experiment to study subjectivity in diagnosis, if any ensuing behavioral changes influence diagnosis rates of ADHD. Using data on over 100 million physician office visits, we compared ADHD diagnosis rates, by day, among children seen by physicians in the 10 weekdays surrounding seven Halloween holidays. The rate of new ADHD diagnosis was 62.7 per 10,000 child-visits on Halloween, compared with 55.1 during surrounding weekdays, a 14% increase. There were no increases in diagnoses of several neuropsychiatric disorders with diagnostic criteria that are less focused on hyperactive behavior. Our findings highlight subjectivity in ADHD diagnosis and support the need to consider external factors that may influence diagnosis.