Data Analysis & Interpretation

This section introduces essential AI-powered analytical tools designed to help behavioural researchers extract meaningful insights from diverse datasets across research methodologies.

For qualitative analyses:

  • AI can extract key themes and trends from large textual datasets.
  • NLP models can analyse sentiment in qualitative responses.

For quantitative analyses:

  • AI can generate visualisations for complex data trends.
  • AI can provide quick solutions to coding errors.
  • AI-powered tools like Julius AI and GenAI platforms may help analyse statistical data.

 

Tools

  • Julius AI (free & paid): https://julius.ai/chat

    Julius AI is a comprehensive data analysis platform that processes multiple file formats, provides visualizations, and executes code in Python and R. The tool offers automatic data correction, PDF information extraction, and structured "Workflows" for common analytical procedures, while employing "Advanced Reasoning" to break complex requests into manageable steps with web search capabilities to enhance analysis. More information is available in this video: https://www.youtube.com/watch?v=OdHLOqxM_XQ&t=41s.

  • Ailyze (paid): https://www.ailyze.com/

    Ailyze can be used for assisted coding, document analysis, and thematic/content analysis. There is also an AI interviewer option for collecting qualitative data, but the free version only offers limited features for qualitative data analysis.

  • GenAI platforms such as ChatGPT, Claude, Gemini and Deepseek can be helpful in qualitative analyses, as well as generating R or Python codes for quantitative analyses.
    • Quantitative analyses in R, Stata, or Python: it can help generate and debug efficient code for complex statistical procedures, creating customised data visualisations, and translating technical outputs into clear interpretations. You can prompt it to recommend appropriate statistical methods based on research questions, generate code for data preprocessing tasks, and write comprehensive documentation (e.g. ReadMe files) for reproducibility. AI assistants can also convert analysis scripts between languages while maintaining analytical integrity if you would like to switch analysis software.
    • More sophisticated AI-powered code editors include GitHub’s Copilot, Cursor and Windsurf. They are also helpful in building web and smartphone applications for your study design

Examples

Research shows that it can be risky for researchers with little expertise in quantitative analyses to use GenAI for such quantitative methods: The study found that ChatGPT 3.5's suggestions for analysis procedures were often general and not clearly illustrated, making it difficult for a naïve user to make informed decisions. It also shows difficulties with proprietary software (SPSS) but more promising results with open-source software (R). The AI can act as a starting point or provide partial solutions, but without a foundational understanding of statistics and software, the naïve user may struggle to evaluate the AI’s output, make informed decisions, and correct errors.

Another research also shows that there could be limits in asking GenAI to produce accurate simple statistical analyses such as ANOVA. When tested on statistical computations, ChatGPT-4.0 initially declined ANOVA requests, suggesting Python as an alternative. Through prompt manipulation, researchers obtained results that matched Stata 15.1's significance determinations in 69% of cases, though the AI couldn't provide actual P values.

For sentiment analysis, a comparison study suggests that LLM models, particularly ChatGPT and fine-tuned OPT, represent a significant advancement in sentiment analysis, especially for health-related survey data. These models offer superior performance and efficiency gains. However, they caution that users should be aware of the limitations, particularly concerning PHI, potential errors with linguistic nuances and specialized terminology, and the necessity for human oversight. Open-source LLMs like OPT provide a promising alternative


In a qualitative study on engagement with research, Researchers employed Ailyze for rapid automated identification of preliminary themes from the dataset. This technological approach was then complemented by manual thematic analysis to refine the findings. According to the paper, the combined analyses allowed for thorough verification of themes, ensuring they accurately reflected the complexities and nuances present in participants' experiences.

A researcher at MIT also compared the performance of thematic analysis conducted by trained graduate students and Ailyze, and found that the AI-powered tool was 75% faster and assessed as producing higher quality output.

Researchers used simulations with ChatGPT and Claude to perform content analyses and narrative analyses and compared them with real human output. Their findings suggest that LLMs hold great promise with the right prompts in coding textual data and generating insightful themes with increased credibility and reduced time.

Researchers also highlight the potential of LLMs as collaborative analytical partners to enhance qualitative by enabling researchers to “converse” with textual data through targeted questioning, revealing patterns and connections that might otherwise remain hidden. AI tools such as Claude can dramatically accelerate traditional tasks like transcription, coding, and theme identification while maintaining the researcher's control over interpretation and conceptual framing. Find out more use examples in the paper; Adam S. Hayes, “Conversing” With Qualitative Data: Enhancing Qualitative Research Through Large Language Models (LLMs).


This article was published on