Personality is a set of psychological traits and mechanisms within the individual that are organized and relatively enduring, and that influence their interactions with, and adaptations to, the intrapsychic, physical, and social environments. The model of personality that has gained the most support in recent decades is the five-factor model of personality. According to the five-factor model, personality is divided into five traits: extraversion, agreeableness, conscientiousness, emotional stability, and intellect. In recent years, numerous papers have been written on the topic of large language models. Large language models are programs that can generate a new set of data based on existing data. Although large language models are still in development, they can be applied as tools in many fields of science and practice. Given their wide range of applications, the question arises as to how much large language models can help in personality assessment. Fifty psychology students studying in the Republic of Croatia participated in this research. The students were presented with a short story about an imaginary person. After reading the story, the students were tasked with assessing the personality of the person using the IPIP-50 personality questionnaire for assessing others. ChatGPT-3.5 and ChatGPT-4 also assessed the personality of the imaginary person, and their results were compared with those of the students. ChatGPT-3.5 and ChatGPT-4 assessed the personality of the imaginary person almost identically. Additionally, ChatGPT-3.5 and ChatGPT-4 rated the imaginary person as more extraverted, less conscientious, and more emotionally stable compared to the psychology students' assessments. The language models did not differ from the psychology students in their assessments of the traits of agreeableness and intellect.