Philosophy:Descriptive Experience Sampling

Descriptive Experience Sampling or DES is a method that aims to uncover the contents of a person's consciousness over the course of short intervals. To do this, practitioners use devices that deliver random beeps. Participants hear these beeps as they go about their daily life. After each beep, they jot down what was in their inner experience in the short moment directly before the beep. This could be a thought, feeling, ‘voice in their head’, or whatever else is present. After a certain number of beeps are collected, participants are given an interview following strict guidelines. DES holds that participants must be trained over the course of multiple days in order to faithfully observe what's in their experience. Findings often differ greatly from participant expectations and sometimes even from scientific consensus.

History

Russell Hurlburt developed the method in the early 1970s. It was refined over the course of the next decades, with the help of frequent collaborators such as Christopher Heavey, Sarah Akther, and Alek Krumm. Hurlburt and collaborators wanted a method to examine inner experience without the memory errors, biases, heuristics, and self-schema based preconceptions that can distort first-person reporting.

First-person methods have had a difficult history. A disagreement between two camps of introspectionists at the beginning of the 20th century led to the field's abandonment by mainstream psychology. An influential 1977 study by Nisbett and Wilson further cemented the notion that first-person reporting is flawed and distorted by memory issues and biases.

Hurlburt and colleagues sought a method that would overcome these limitations. DES complies with Nisbett and Wilson's oft-overlooked recommendations for how first-person reports could be more accurately obtained. These include 1) interrupting a process at the moment it is occurring, 2) alerting subjects to pay careful attention to their cognitive process, and 3) coaching them in introspective procedures.

Hurlburt's research started with the use of the beeper device in naturalistic settings. Originally he gave participants a questionnaire with a limited range of options. This facilitated quantitative comparison. But reportedly, Hurlburt grew frustrated at the limitations this placed on unveiling experience. He moved towards more in-depth qualitative interviewing. He studied the work of Husserl and Heidegger and drew inspiration from phenomenology.

When first refining the method, Hurlburt at first sampled himself. He then concluded that it would be better not to use himself as a subject. Phenomena that he observed in himself he might more easily attribute to others. For the next 25 years or so he refused to participate in DES as a subject until the urgings of his students convinced him to try.

Method

Procedure

DES uses a device that delivers random beeps to participants throughout the day. It can be a specialized device or a programmed smartphone. The beeps are delivered through an earpiece—to increase the immediacy with which participants can observe their experience. In a typical procedure, participants receive six beeps a day. Sampling occurs in the participant's everyday environment to increase ecological validity. They could be doing laundry, having lunch with friends, driving to work, or whatever else they would typically do.

This shows the DES process. Participants receive six beeps a day. They jot down their experience of the moment before the beep. Within 24 hours of these beeps being collected, they are given a one-hour interview. This entire process is repeated, usually for six or more days.

After each beep, participants jot down what was in their inner experience directly before the beep. Not after or during, but directly before. This is sometimes referred to as the “millisecond” before the beep, or the “last undisturbed moment”. These phrasings are both somewhat inaccurate, but the goal is to convey to the subject that they should be as precise as possible in their descriptions.

After six beeps are collected, participants are given a one-hour interview. This interview is delivered within 24 hours of beep collection, to reduce errors of memory recall. The premise of one-hour interviews is to not exhaust the patient. Any beeps not discussed within the hour are discarded. Participants are told that they can choose not to report any beep that is too private.

The entire process of beep collection and interviewing is repeated, usually for around six days but often longer. The first day of sampling and interviewing is always considered training and discarded from further data analysis. Other days may be discarded as well if interviewers judge that the participant is not yet adequately trained. DES holds that observing experience is difficult and the sustained efforts of both the participant and interviewer, as “co-investigators,” are required. What follows is an example of an admissible DES sample, which researchers compile based on the participant's interview responses.

Steven was pacing around his condo engaged in a mental argument. At the beep he was innerly saying the word “whatever” to himself in his own voice, as if directed at the person he was mentally arguing with. He was also aware of a sense of frustration and an accompanying sensation of heat and outward-radiating pressure behind his ears and eyes. Simultaneously, he was also aware of a “frenetic” restless energy in his arms and legs that made him feel like he had to be moving.

After samples are collected, they can be coded. This sample, for example, could be coded as containing inner speaking (the “whatever” element), feeling (the sense of frustration), and sensory awareness (the restless energy). After coding, intrasubject, intersubject and intergroup analysis can also be performed.

Interview guidelines

The interview procedure is detailed in books like Exploring Inner Experience: The Descriptive Experience Sampling method. It is also available in an interactive website and a video series. DES interviews follow rigorous guidelines. A core component is “bracketing presuppositions”. Interviewers must leave behind their notions of what they think experience is like. A participant's experience may be very different from their own. Participants are also trained to bracket their own presuppositions. They might at first have preconceived notions of their experience that prevent careful observation.

DES literature contains examples of participants originally mistaken about their experience. For example, one participant, Donald, prior to DES, described his experience as mostly consisting of anxiety. But DES Sampling revealed that in a good deal of his samples he was angry, specifically at his children. Donald denied this theme until shown his samples. Hurlburt interprets that retrospective self-accounts are often incorrect. Presuppositions can be difficult to overcome.

In order to reach accurate descriptions without biasing participants, DES uses what Hurlburt calls “open-beginninged probes.” One such question could be: ‘What, if anything, was in your experience at the moment of the beep?’ Other phrasings like "what were you thinking about at the moment of the beep" could bias participants—for example suggesting that they need to report something that qualifies as “thinking.”

Interviews should avoid generalizations and guide participants toward the concrete. For example, a participant might answer, “I always am talking to myself.” This is not admissible for DES. The goal is to find what was in experience specifically at the instance before the beep.

If questions aren't content-neutral, interviewers should give multiple options. For example, if a participant describes a mental image, the next step could be eliciting greater precision. Questioning could proceed: ‘Were there borders around the image? Or no borders? Or you’re not sure?’ Opportunities should be given for participants to revise their story or change it completely. Interviewers pay attention to “subjunctification.” This includes hesitation, words like ‘umm,’ ‘like,’ ‘I guess,’ and ‘I suppose’. These can indicate the participant's doubt and their removal from direct experience.

The goal of DES is not to force an accurate description for every sample. Often, samples are inconclusive. The goal is to train participants so they can be more sensitive to their experience on subsequent days. Each day of interviewing can be considered training for the next.

Validity

Claims to validity

Hurlburt and Heavey write that DES follows ‘idiographic validity’. By this, they mean that we can only judge the validity of DES for one participant at a time. Researchers approaching the method should ask if they are convinced by the argumentation behind the method's guidelines. They should then ask if the researcher and participant in question complied with these guidelines.

Validity studies can also be performed. One study looked at the interobserver reliability of interviewing and coding. Two researchers independently interviewed DES participants and coded their experiences. They compared these codes to see if they matched and found high reliability.

DES samples can also be checked with other observables. Hurlburt and Heavey refer to this as situating DES in a “nomological network.” This means an interlocking system of observables that can help build up validity. These could be first-person or third-person observables.
For example, one DES participant, Fran, did not have any figure / ground distinction in her mental imagery. No part of her mental images appeared closer or better in focus. Hurlburt was surprised by this and sought another way of testing. He showed Fran classic figure/ground illusions, for example an image that simultaneously shows faces and a vase. Fran reported seeing both the faces and the vase at the same, with no alternation between them. The third-person measure—Fran's response to the illusion—was used to corroborate Fran's first-person reports.

Other studies can be done incorporating third-person data, for example neuroimaging studies. For example, samples of inner speaking while in an MRI scanner correlated with activation in classic speech processing areas including the left inferior frontal gyrus.

Other evidence for validity could come from participants being helped by the DES process. For example, after sampling, Fran was able to control intrusive thoughts better. Other participants have gained better clarity over their inner life and relationships. Outcomes like these can be part of the interlocking system of observables, even if they aren't, in themselves, proof of validity.

Criticism

For some, DES, despite its efforts, doesn't overcome issues of first-person reporting including biases and memory constraints. For example, Eric Schwitzgebel sees first-person reporting as still too subject to distortion. Hurlburt and Schwitzgebel have addressed these criticisms in a book where Hurlburt conducts DES with a participant. Hurlburt and Schwitzgebel then discuss the potential limitations of the method.

Another line of criticism is that DES doesn't go far enough in uncovering certain aspects of experience. For example, its narrow temporal scope might leave out certain temporally extended phenomena. Or its lack of directing participants' awareness could mean it misses certain nuances of experience.

Hurlburt acknowledges that DES samples can be incomplete and may miss some elements of experience. To him, being confident of reported elements is more important than capturing all possible elements in experience.

Findings

Importance of the individual

A key insight from DES research is how variable individual experience can be. Different participants can have very different kinds of conscious experience. Hurlburt writes:

I have sampled with some people whose inner experience is characterized almost exclusively by inner speech; with others whose inner experience is characterized almost exclusively by images, or by sensory awareness, or by unsymbolized thinking, or by feelings; with others whose inner experience is characterized by a combination of all those; with some whose inner experience is characterized by many simultaneous events; with others whose inner experience is characterized almost exclusively by one event at a time; and so on. So, yes, I think people are importantly different when it comes to inner experience.

DES has been used to form generalizations about groups, for example regarding various mental disorders. But researchers emphasize that beyond such generalizations, we should retain the importance of individual experience. Researchers call this an 'idiographic' focus.

Hurlburt gives the example of a participant, RD, who described some thoughts as being “solid” and some as being “light.” The goal of interviewing was to determine whether this distinction represented a salient aspect of RD's experience or if it was merely a quirk of his descriptions. After questioning Hurlburt concluded that this division was indeed a salient feature. A light thought meant RD was “not deeply focused on the thought or working on the thought.” A solid thought, in contrast was “heavily concentrated, deeply focused”

This division of experience into 'solid' and 'light' is not common. Particularities of individual experience can emerge through sampling. This doesn't, however, preclude further sampling with other participants from revealing similarities.

Five frequent phenomena

Once interviewers recognize that their primary focus is on individual experience, they can build up an understanding of consistent features across participants. For example, research has identified five particularly frequent phenomena: sensory awareness, inner speaking, images, feelings, and unsymbolized thinking. One study looked at the inner experience of 30 college students to estimate frequencies of these phenomena. Frequency varied substantially for each individual. For instance, nine participants had no samples of sensory awareness throughout their ten samples. One participant had sensory awareness in 100% of samples.

What follows is a description of each of these five frequent phenomena, with examples from other studies.

Sensory awareness

This denotes paying particular attention to sensory aspects of the world or one's own body. For example, a participant might notice, “a shiny blueness, feel the coldness of an iced drink, hear a feature of a friend’s voice, or feel a muscle twinge.” To be coded as sensory awareness, the participant should be focused on these aspects—they're not merely passive in the background. Hurlburt gives the example of someone skillfully driving a car but without their attention directed at their surroundings. He compares this to someone driving a car with their attention on the “particular yellow color of the yellow line.” The latter would be coded as sensory awareness.

As noted, there is great intrasubject variability for sensory awareness. Some participants can hardly ever be attuned to sensory aspects of their environment. Some participants can have sensory awareness in nearly every sample, or multiple different instances in one sample.

Variations in sensory awareness may play a role in psychopathology. From one study, people with autism could have sensory awareness occupy 100% of their samples, to the exclusion of all other phenomena. People with schizophrenia can have distortions in their sensory awareness—vision can be scratched, warped, and shuffled.

Inner Speaking

Inner speaking denotes what some might call ‘self-talk’ or a ‘voice in their head’. But whereas some use these phrases metaphorically, for some they can be quite literal.

Inner speaking can take different characteristics. It is often in the participant's own voice, although can take on characteristics of people they know. Sometimes tone, pitch, and auditory characteristics are clear and sometimes not. Sometimes the voice occupies specific regions, often in participants’ heads. Sometimes this localization is more metaphorical, and sometimes there is no localization at all. Inner speaking can sometimes fully express thoughts. At other times, it may only emphasize certain words—in what can be called partially worded speech

In the aforementioned study on the frequent phenomena of experience, the presence of inner speaking correlated with a decrease in psychological distress. Inner speaking might then have a role in lessening psychological distress.

Images

Sometimes called “inner seeing,” this refers to mental images. These are different from participants observing the world in front of them. A participant could for instance be physically in an office but have a mental image of a palm tree on a beach.

DES researchers have observed a good deal of variability regarding mental images, both between samples and between participants. Sometimes images can be clear, and sometimes vague and ill-defined. Sometimes they can appear with borders around them like a photograph. Sometimes they can appear as if the participant is present ‘in the scene.’ Sometimes they can be in color. Sometimes they can be in black and white.

Hurlburt and Heavey speculate that inner seeing is a skill that can take time to acquire and can also be lost. They illustrate this with an example from a 9-year-old boy. At the moment before the beep, he had a mental image of a hole he had been digging in real life. When asked if it was like the real hole, the boy replied “Yes […] except that the real hole has more toys in it. If you had beeped me in a couple minutes, I would have had time to finish the picture.” The authors infer that creating detailed mental images is a skill that adults might take for granted but takes practice to develop. They compare this with sampling from a 81-year-old women whose mental images were entirely in black-and-white or brown-and white. They speculate that detailed, colored inner seeing is a skill that for her has declined with age.

Feelings

This code denotes awareness of emotional experiences like sadness, happiness, worry, and frustration. In DES, 'feelings' does not refer to physical sensations, which are coded as sensory awareness. It is also distinguished from “emotions” in that emotional processes may be ongoing but not always in direct experience. In some samples, for example, participants have behavioral manifestations of anger, but are not angry at that exact moment.

As with other codes, there is a great deal of variability between participants. From the study with 30 college students, five of them had no instances of feelings in their samples. One of them had feelings in 90% of samples. Contrary to some theorizing, feelings are not always present in experience, at least from the ability of DES to discern.

Sometimes feelings are clear and sometimes not. Sometimes multiple feelings can be ‘mixed’ together simultaneously—for example, happiness with a tinge of sadness. Sometimes feelings can have a clear bodily location and sometimes not.

Similarly to mental images, Hurlburt speculates that experience of feelings may also be a skill that takes time to develop. He uses as an example a sample with an 11-year-old who had watched a show where one of her favorite TV characters had died. At the moment right before the beep she was repeating to herself “I’m sad, I’m sad, I’m sad…” but “was not actually feeling sad at that moment.” The verbal repetition may have served to coordinate the skillful ability of feeling sadness.

Unsymbolized thinking

This code refers to thoughts that have clear, differentiated content but no discernible features that “carry” that content: no images, no words, no other kinds of symbols. For example, in one sample Abigail is wondering whether her friend will pick her up in his car or his pickup truck. This content is clear. But there are no words or images accompanying it, merely the content. Some consciousness scholars deny the existence of unsymbolized thinking, but this is often based on self-initiated and self-directed introspection without a defined method. Many DES participants have unsymbolized thinking without beforehand recognizing it as a feature of their experience. Since content can be clear, unsymbolized thinking might at first be mistaken for other phenomena like inner speaking. Even after decades of research, Hurlburt was unable to recognize unsymbolized thinking in his own experience before DES sampling revealed it as a common feature.

Use for psychopathology

DES's focus is first on individual samples and then on individual people. Commonalities may, however, emerge for certain groups. This includes for people with psychiatric diagnoses. Many of these studies have small sample sizes and could be considered exploratory. But some clearer findings, with replications, have emerged—for example regarding bulimia nervosa.

Bulimia nervosa

With bulimic participants, multiple DES studies have found what researchers have termed ‘fragmented multiplicity.’ This means that bulimic individuals can have multiple elements in experience at the same time. These could be simultaneous images, feelings, or other kinds of experience. There may be over a dozen simultaneous elements at once. Depending on the participant, the proportion of samples with multiple fragmented inner experience ranged from 44% to 92%.

These simultaneous elements may or may not be focused on body image issues. This shows that bulimia may be characterized more by the process of experience than the content. An example of a sample showing fragmented multiplicity:

Sample 5.2. Jessica was looking at her digital camera display, seeing a photo of her and her boyfriend taken on a recent trip to Chicago. While seeing this photo, she was also innerly seeing at least five separate, simultaneous, overlapping visual scenes of places she had visited in Chicago. These inner seeings were fuzzy or indistinct, and were apprehended as if looking at snapshots – the scenes had edges, for example.

Simultaneously, she was innerly seeing herself and her boyfriend standing close together at the kitchen sink. In this seeing, which was somewhat clearer than the Chicago scenes, Jessica was on the left, the boyfriend on the right, and both were seen from the back. This was a re-creation of an event that had actually taken place, but viewed from behind her, an obviously impossible perspective for her to have taken in reality.

Simultaneous she was feeling happy, apprehended as a volleyball-sized sensation deep in her stomach but also all over her stomach.

In this one sample, Jessica had multiple instance of inner seeing and a simultaneous feeling. These elements didn't occur sequentially over a lengthy period of time. They were all present at once.

Another feature of bulimic experience is elements leaving direct consciousness but still lingering, as if they might reoccur. Participants have different terms for this, for example calling these elements “tails”—like the visible tails of fish hiding under rocks.

Regarding fragmented multiplicity and 'tails,' participants are generally unaware of these features before sampling. These features have also not been described elsewhere in literature. They show that, if valid, DES may be able to uncover features that other methods miss. These features could then potentially be useful in diagnosing and treating mental illnesses.

Autism spectrum

One small study looked at three adults with Asperger's syndrome, a diagnosis now recognized as belonging to the autism spectrum. For one participant, experience was unclear, but for the remaining two, visual images were the dominant feature of their experience. Mental images or visual sensory awareness occupied up to 100% of participant samples. Cognitive processes like problem solving manifested through images. One participant also described these images as forming “the shape of [his] thoughts”. For example, in one sample, he was looking at a brick wall and was visually focused on three or four bricks. He described his thoughts as having ‘taken the shape of’ the bricks. His awareness was completely occupied by them and nothing more.

Anxiety disorder

In a small sample of five individuals, some characteristics emerged. Compared to individuals without psychiatric diagnoses, those with anxiety disorder had a relatively high frequency of indeterminate visual images (between 8 and 25% of their samples). Indeterminate images are images with little or no clarity, color, or detail. Another feature was a relatively high frequency of worded thinking—words being present without being innerly spoken or heard.

Participants also commonly had repetitive rumination and critical thoughts—either critical of the participant or of others. Another feature was what Hurlburt calls the “’doing’ of understanding and the ‘happening’ of speaking”. Understanding others required effort and concentration, but speaking simply occurred undirected and effortless. For people without anxiety, often the opposite is true. Speaking can take more effort than understanding.

Schizophrenia

Sampling with schizophrenic participants revealed some commonalities. Images were experientially important, as was color in these images. These images could exist more concretely than for non-schizophrenic participants. Images or visual sensory awareness could often be "goofed up"—scratched, warped, or otherwise distorted. Hurlburt speculates that schizophrenia may be more a disorder of distorted perception than of disorganized association.

Another inference based on interview analysis was that decompensating schizophrenics (in the midst of severe episodes) may sometimes have no inner experience at all.

Major depressive disorder

DES sampling results with depressed people have been inconsistent. One study found that depressed people had a much greater frequency of unsymbolized thinking than non-depressed people. Clarity of thinking decreases with depression. Another study did not replicate this. This study also did not find statistically significant differences in depressive symptomology between depressed and non-depressed people. This means that the depressed group did not have a significant increase of samples with depression, anxiety, fatigue, body discomfort, negative feelings, negative content, or reduced positive feelings.

One study looked at different response styles of depression—rumination vs. distraction. Those with a ‘rumination’ response style cope with negative content by repeatedly mulling it over. Those with a ‘distraction’ response style cope by distracting themselves with other thoughts. People with a ‘rumination’ response style had a higher frequency of unsymbolized thinking and feelings.

Reasons for unclear findings could be that depression is a broad symptom cluster making it difficult for commonalities to emerge. Further division of different subtypes (as in the response style study) could be useful for psychopathology. Another reason could be that identification of depression can depend on self-schemas that may not reflect experiential particularities. Depression questionnaires rely on memory and retrospection. A participant's self-schema could influence how they answer these questionnaires. DES samples minimize memory demands and give a different picture of experience. Another reason for unclear findings could be that DES, with its narrow temporal scope, may not uncover all aspects of depression.

Use with neuroimaging

DES has been used in combination with neuroimaging in pursuit of bridging an understanding of the mind with its physical substrate. One study found that when DES revealed samples of inner speaking in the MRI scanner, classical language areas of the brain were activated.

A further study showed that while language areas were activated during spontaneous inner speaking, these were different areas than those active during tasks. Participants were scanned at resting state—not given any particular instruction. Of collected samples, those that included inner speaking were analyzed. Brain activation for this inner speaking was different than for inner speaking typically elicited in fMRI studies, where participants follow instructions for specific tasks. The authors conclude that we should be wary of extrapolating from task-based fMRI to infer about natural experience.

Another study analyzed what DES samples can say about ‘resting state’ experience. DES samples were collected when participants' default mode network was active. The default mode network is commonly seen as responsible for our 'resting state.' The study found a variety of experience. Participants could have very different frequencies of the five frequent phenomena (described above). This is counter to scientific literature describing resting state consciousness as a unified phenomenon. The authors also proffer to more accurately describe resting state as “unconstrained activity.”

In another study, researchers rated DES samples in the scanner as either being internally or externally directed (or both simultaneously). Internally directed samples correlated with default mode network activity. A model was then able to predict raters choices based on neural data. This demonstrates a step towards being able to describe consciousness based on neuroimaging.

Other findings

Left-handers

In a study of left-handed participants, a number of salient features emerged, including words experienced without semantic significance. Verbal content could be present in experience, without accompanying meaning. Participants could either be looking at these words, visually imagining them, or innerly speaking them. The focus was on visual or auditory aspects of these words rather than what they represented. This may have to do with left-handers' differing lateralization of brain function.

Absence of experience

In quite a few DES samples, participants appear to have no experience at all—even if experience is defined in the broadest of ways. To the best of their recollection, they have no thoughts, emotions, sensation or anything else that could constitute experience. For example, in one sample a participant, Ben, has no experience and describes this like having a “void within”.

Many philosophers of consciousness argue that this is impossible, and that we always have some type of experience. Hurlburt argues that the process of questioning leading to a conclusion of absent experience is thorough. Still, DES cannot completely rule out some minimal experience.

Some DES research has hypothesized absence or near absence of experience for individuals—for example for one participant with autism, or for shizophrenic participants when symptoms become severe.

Silent reading

A study found that, contrary to the beliefs of certain theorists, very few samples of reading involved inner narration of the text—only three percent in fact. Other elements were common, for example visual imagery. According to the authors, this shows that experience can be different than participant presuppositions and the presuppositions of theorists.

References

• Brouwers, V. P., Heavey, C. L., Lapping-Carr, L., Moynihan, S. A., Kelsey, J. M., & Hurlburt, R. T. (2018). Pristine inner experience while silent reading: It's not silent speaking of the text. Journal of Consciousness Studies, 25(3-4), 29–54.
• Dainton, B. (2000). Stream of consciousness (Rev. ed.). Routledge.
• Dickens, Y. L., Raalte, J. V., & Hurlburt, R. T. (2018). On investigating self-talk: A Descriptive Experience Sampling study of inner experience during golf performance. The Sport Psychologist, 32, 66–73. https://doi.org/10.1123/tsp.2016-0073
• Doucette, S. A. (1992) Sampling the inner experience of bulimic and other individuals (231) [Doctoral dissertaion, University of Nevada, Las Vegas]. UNLV Retrospective Theses & Dissertations.
• Fernyhough, C. (2016). The voices within. Profile Books.
• Gunter, J. D. (2011) Examining experience in depressed and nondepressed individuals (1215) [Doctoral dissertation, University of Nevada, Las Vegas.] UNLV Theses, Dissertations, Professional Papers, and Capstones.
• Heavey, C. L., & Hurlburt, R. T. (2008). The phenomena of inner experience. Consciousness and Cognition, 17(3), 798–810. https://doi.org/10.1016/j.concog.2007.12.006
• Heavey, C. L., Hurlburt, R. T., & Lefforge, N. L. (2012). Toward a phenomenology of feelings. Emotion, 12(4), 763–777. https://doi.org/10.1037/a0026905
• Heavey, C. L., Lefforge, N. L., Lapping-Carr, L., & Hurlburt, R. T. (2017). Mixed emotions: Toward a phenomenology of blended and multiple feelings. Emotion Review, 9(2). https://doi.org/10.1177/1754073916639661
• Hurlburt, R. T. (1980). Validation and correlation of thought sampling with retrospective measures. Cognitive Therapy and Research, 4(2), 235–238.
• Hurlburt, R. T. (1993). Sampling Inner Experience in Disturbed Affect. Plenum Press.
• Hurlburt, R. T. (2011). Investigating pristine inner experience: Moments of Truth. Cambridge University Press.
• Hurlburt, R. T. (Date accessed: April 24, 2023) DES-imp: Interactive DES example and training. http://hurlburt.faculty.unlv.edu/desimp/labs/lab0/lab0.html
• Hurlburt, R. T. (2022) Do I really have internal monologue? (Reality TV about inner experience). (Date accessed: April 25, 2023). http://hurlburt.faculty.unlv.edu/lena/do_I_have_internal_monologue_sampling.html
• Hurlburt, R.T. & Akhter, S.A. (2006). The Descriptive Experience Sampling method. Phenomenology and the Cognitive Sciences 5, 271–301. https://doi.org/10.1007/s11097-006-9024-0
• Hurlburt, R. T., Alderson-Day, B., Fernyhough, C., & Kühn, S. (2015). What goes on in the resting-state? A qualitative glimpse into resting-state experience in the scanner. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01535
• Hurlburt, R. T., Alderson-Day, B., Kühn, S., & Fernyhough, C. (2016). Exploring the ecological validity of thinking on demand: Neural correlates of elicited vs. spontaneously occurring inner speech. PLoS-ONE, 11(2). https://doi.org/10.1371/journal.pone.0147932
• Hurlburt, R. T., Happé, F., & Frith, U. (1994). Sampling the form of inner experience in three adults with Asperger syndrome. Psychological Medecine, 24, 385–395.
• Hurlburt, R. T., & Heavey, C. L. (2002). Interobserver reliability of Descriptive Experience Sampling. Cognitive Therapy and Research, 26(1), 135–142. https://doi.org/10.1023/A:1013802006827
• Hurlburt, R. T., & Heavey, C. L. (2006). Exploring inner experience: The Descriptive Experience Sampling method. John Benjamins. https://doi.org/10.1075/aicr.64
• Hurlburt, R. T., Heavey, C. L., & Kelsey, J. M. (2013). Toward a phenomenology of inner speaking. Consciousness and Cognition, 22(4), 1477–1494. https://doi.org/10.1016/j.concog.2013.10.003 
• Hurlburt, R. T., & Schwitzgebel, E. (2007). Describing Inner Experience? Proponent Meets Skeptic. MIT Press.
• Hurlburt, R. T., & Sipprelle, C. N. (1978). Random sampling of cognitions in alleviating anxiety attacks. Cognitive Therapy and Research, 2(2), 165–169.
• Jones-Forrester, S. (2009). Descriptive Experience Sampling of individuals with bulimia nervosa (1212) [Doctoral dissertation, University of Nevada, Las Vegas]. UNLV Theses, Dissertations, Professional Papers, and Capstones.
• Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295x.84.3.231
• Petitmengin, C. (2006). Describing one's subjective experience in the second person: An interview method for the science of consciousness. Phenomenology and the Cognitive Sciences 5, 229–269. https://doi.org/10.1007/s11097-006-9022-2
• Robinson, W. S. (2005). Thoughts without distinctive non-imagistic phenomenology. Philosophy and Phenomenological Research, 70, 534–561.
• Scott, T. A. (2009). Evaluating the response styles theory of depression using Descriptive Experience Sampling (56) [Master's dissertation, University of Nevada, Las Vegas]. UNLV Theses, Dissertations, Professional Papers, and Capstones.
• Sutton, J. (2011). Time, experience, and Descriptive Experience Sampling. Journal of Consciousness Studies, 18(1), 118–29.
• Vásquez-Rosati A. (2017) Body awareness to recognize feelings: The exploration of a musical emotional experience. Constructivist Foundations, 12(2), 219–226.
• Vermersch, P. (1999) Introspection as practice. Journal of Consciousness Studies, 6(2–3), 17–42.