In this post, I discuss how to use ChatGPT in social science research.

The take-away message is: The AI is the assistant and you are the director.

Many AI tools for social scientists

There are many AI tools out there, including ChatGPT. Otter.ai does transcripts. Notion.so integrates notes (including transcripts). Consensus.app is like Google Scholar but it answers questions.

The best way to think about how to use AI tools is this: AI is the assistant and you are the director. As the director of AI assistants, you can choose which one to use, in what capacity, and at any time.

As a social scientist, you are the director of AI research assistants.

See: Artificial Intelligence (AI) and the Social Scientist

Right now, ChatGPT is primitive. But, ChatGPT and its relations will improve, fast and dramatically, over time. The limits we see now will disappear within five years or so (perhaps sooner).

ChatGPT limits

First and foremost, we must recognize the limits of ChatGPT for writing a research paper.

ChatGPT is text-based, meaning it can only read text. The term for entering information is a “prompt.” For how ChatGPT works, see OpenAI.

ChatGPT, even in its latest iteration (ChatGPT 4) has severe limits, and chief among them is that it will simply write things that are not true. The disclaimer at the bottom:

“ChatGPT may produce inaccurate information about people, places, or facts. ChatGPT Mar 23 Version”

The parlance is “hallucination,” but this is corporate brand-speak for what is really going on: the confident presentation of wrong information. ChatGPT will invent book titles and authors that never existed and claim that they are part of the literature. It will confidently and incorrectly define a concept and wrongly portray the relationship between concepts.

ChatGPT writing, unedited, cannot rise to the level of sophistication necessary for publishing a good article in a journal.

(it can probably, right now, write a paper to publish in a low-visibility, peer-review challenged journal, and certainly in predatory journals)

No matter the tone or style you set in the prompts, it cannot get past the low-level writing. Even though it can insert citations, the first limitation – it makes stuff up – renders the service too unreliable to write a full paper with it. (ChatGPT 4 is better than ChatGPT 3, but it still can make stuff up)

A few more limits.

• It only provides information that existed as of September 2021. It does not know anything afterwards.
• More importantly, it cannot crawl the web. If you want it to summarize a text from a website, you have to copy and paste it into the ChatGPT box.
• There is a character limit, but ChatGPT does not provide it. It tells you to keep it at a “reasonable length.”

How to use ChatGPT?

Given these limits, ChatGPT is a useful AI-assistant, albeit one in which you must check their work. You have to be a diligent director.

Here are five ways to use ChatGPT. There are more, but I will provide only examples of these. 

Brainstorming

Getting the thought-process started is perhaps the best feature of ChatGPT. Enter a topic and ask for ideas, e.g. “why is the sky blue?” or, “We have a meeting on improving our knowledge of power analysis in psychology. What topics should we cover?” and so on.

Summaries of text

Enter in any text and ask for a summary. Yes, you can ask to provide the summary in the form of a poem, etc. But for research purposes, you can ask it to provide the summary in a variety of styles and tones, e.g. “You are now ProfessorGPT. Please summarize the following text as a professor would in delivering a talk to their peers.” Or, “please explain this text as if I am in my first year of high school.” 

Text to edit

You can ask ChatGPT to write a paper on the relationship between concepts that is prepared for an academic journal in the social sciences. It will do so, complete with subheadings, but the writing has limits (see above). From here you can edit the text, put in citations, and other such tasks until the paper starts to provide knowledge that readers can use.

Simple analysis of data from .csv files

You can input data only in the form of a .csv file and ChatGPT can read it and provide a table or a list of the data. You can, in the same prompt, ask it to analyze the data. 

Creating tables

There are many prompts one can use, and new ones are being discovered all the time. You can ask ChatGPT for these prompts, or you can Google them. One powerful prompt is to provide a table. For example, “Please analyze this interview present five main themes. Place the themes in a table, where the first column is the theme and the second is a quote from the interview that supports the theme.”

Coding

One of the most powerful uses of ChatGPT is to assist in coding. It knows a variety of coding languages, including R, but also Python, etc. It may provide wrong syntax, but you can put the error messages into ChatGPT and ask it to fix the code. At this point, it is a painstaking process, but future versions of ChatGPT will make the experience better.

An example of how to use ChatGPT

In this example, I use ChatGPT for everything. I conceived of an example, and asked ChatGPT 3.5 and 4 to perform various tasks. I used R for the data analysis.

I want to understand differences in age groups on Time Perspective in Poland. 

I used the European Social Survey for 2018 (ESS 2018). These data are free to download and use.

Using a survey item in ESS 2018: “Do you generally plan for your future or do you just take each day as it comes? Please express your opinion on a scale of 0 to 10, where 0 means ‘I plan for my future as much as possible’ and 10 means ‘I just take each day as it comes’.”

Using ChatGPT, I eventually received R syntax that I can use. I’m still learning how to use ChatGPT to provide syntax in R (subject for a later post). This is an example of a prompt:

“I need syntax for a research project. The syntax is in R. The working directory is “C:\Users…

The dataset is found through this path: “C:\Users….csv”

Please recode “agea” such that all values less than 20 and greater than 80 are treated as missing cases. Call the new variable, “agea_20to80”. After producing this, please provide a histogram of this variable.”

Specifically, I asked it to provide the syntax for R to calculate the average score of “plan for the future” by age brackets and save it as a .csv file in my computer. 

I received usable code and eventually got a .csv file. Next, I created a prompt for ChatGPT.

“You are now ProfessorGPT. I will provide some information. Please write me a short paper based on the information. Here is a .csv file.

“agea_20to80_recode”,”mean_plnftr”

20-30,4.29437229437229

31-40,4.40073529411765

41-50,4.75925925925926

51-60,5.33018867924528

61-70,5.77323420074349

71-80,7.16153846153846

The values in column one are age brackets. The numbers are average scores from the survey item from the 2018 European Social Survey that reads: ” “Do you generally plan for your future or do you just take each day as it comes? Please express your opinion on a scale of 0 to 10, where 0 means ‘I plan for my future as much as possible’ and 10 means ‘I just take each day as it comes’.”  Please write a short paper with a title, an introduction, a research question, some cited literature, and explanations for age differences in the average scores. Please go in-depth on age brackets by citing the data, noting which have higher scores and which have lower scores. At the end, provide the works cited.”

Summary

It worked.

ChatGPT 3.5 and 4 both provided a paper based on my prompt. In some iterations, ChatGPT 3.5 provided a wrong interpretation of the .csv data. However, ChatGPT 4 produced the correct interpretation every time.

Conclusion

In this post, I explored how social scientists can use ChatGPT, despite its limitations, in writing a research article. ChatGPT is currently primitive, but is expected to evolve, dramatically, over time.

I discuss ways to use ChatGPT effectively: brainstorming, summarizing text, creating text to edit, simple data analysis from .csv files, and creating tables, and coding for languages like R. By being diligent and checking the AI-assistant’s work, researchers can make the most of ChatGPT’s capabilities.

Because it will make things up, you have to be a diligent, micro-manager of a director. AI is not yet reliable enough to trust it on faith.

As an example of how to use ChatGPT in social science research, I demonstrated how I used ChatGPT to analyze differences the European Social Survey for 2018. Using a combination of ChatGPT and R for data analysis, I obtained useful insights.

How can social scientists use ChatGPT in their research. By remembering this:

The AI is the assistant and you are the director.