The Structure of Speech
Processing speech is complex:
•Environment can interfere with speech signal
•Variability in speech signal
No one-to-one correspondence between the acoustic stimuls and the speech sounds we hear.
How do recognize sounds in a way so that we perceive hearing a stable set of phonemes?
Articulatory Phonetics
the study of the pronounciation of speech sounds
For Consonants:
Place of articulation
Bilabial: lips ([b], [p], [m])
Alveolar: alveolar ridge ([d], [t], [s], [z], [n], [l])
Velar: back of the mouth ([k], [g], [ng])
Manner of articulation
Fricatives: obstruction of airflow, but some airflow ([f], [s], [z])
Stops: obstruction of airflow and then release ([b], [p], [k])
Affricate: Stop followed by fricative ([ch], [j])
Voicing
Voiced vs voiceless: (z/s, b/p, v/f)
Vowels
Articulatory gestures are mainly associated with the position of the tongue
Part of tongue (pit, putt, put)
Height of tongue (pit, pet, pat)
Acoustic Phonetics: Studying the acoustic signature of speech sounds
Problem: No parallel between phonetic segments and letters
How do we study acoustic energy of speech sounds?
Spectograms: representations of sound frequency by time
Interpreting spectograms
Formants: Bands representing sounds occuring at particular frequency levels
Aspects of Formants
Formant Transitions: Changes in the formant frequency
Steady state: No changes in formant frequency
Acoustic properties of the speech signal
Parallel transmission
The speech signal is continuous
Context-conditioned variation
The spectographic appearance of a phone is related to the context and not to that particular phone in other contexts
Coarticulation: We are producing more than one speech sound at the same time
The problem of invariance: There is no one-to-one correspondence between acoustic cues and perceptual events
The perception of isolated speech segments
Categorical perception
• Can we discriminate between two sounds/colors/etc. that are very similar?
• In order to process speech, we need to impose categorical identification on the incoming sounds.
Test of categorical perception: /ba/ vs. /pa/
Lisker & Abramson (1960)
• Synthesized speech sounds
• Varied VOT from -0.15 to +0.15 secs
Does categorical perception hold for nonspeech sounds as well?
Mattingly, Liberman, Syrdal & Halwes (1971)
• Synthesized speech syllables:
• Only presented the second formant, or reversed the sound, or removed the transition.
• Found no categorical perception.
• Suggests that the formant transitions are important for categorical perception
What about vowels?
• Vowel perception is continuous and noncategorical
• Why? longer duration of cues
The role of attentional processes
Mann, Madden, Russell, & Liberman (1981).
Binaural procedure
Base presented to one ear, transitions presnted to other ear.
The conditions to process speech in this phonetic mode depend on attention and not just the acoustic cues
The perception of continuous speech
Acoustic structure varies greatly as we speak.
Pollack and Picket (1964)
• Processing words out of context
• Words were identified correctly only 47% of the time
Acoustic information alone may not be enough to identify speech sounds. We need context
The role of Semantic and Syntactic Features
Millar & Isard (1963).
Present three different types of sentences
1) grammatical
2) anamalous, preserving grammatical word order
3) ungramatical
1) Accidents kill motorists on the highways
2) Accidents carry honey between the house
3) Around accidents country honey the shoot
Illusions
Miller (1956) Processing sounds through low-pass filter
Pooh kluss free soub eatwull size
Two plus three should equal five
Warren (1970)
Phonemic Restoration
The state governors met with their respective legi*latures convening in the capital city.
Warren & Warren (1970)
It was found that the *eel was on the axle
It was found that the *eel was on the show
It was found that the *eel was on the orange
It was found that the *eel was on the table
Acoustic Phonetics lab review
da ta ba pa dee dew day daw doe
A) Is there a definable break between the consonant and the vowel? (i.e. can you select a portion of the consonant so that you can hear the consonant, but not the vowel?)
Coarticulation:
We produce more than one speech sound at a time
B) Identify the difference in Voice Onset Time (VOT) between "da" and "ta" and between "ba" and "pa".
C) How do the first and second formats differ across the sounds "dee, dew, day, daw, and doe"? Is there any feature of the "d" sound that is consistent across the sounds?
Context conditioned variation
: the spectographic appearance depends on the speech context
2) My dog is lying under the bed
A) Can you identify the breaks between words? How much time elapses between words?
B) How does the "d" differ in "dog", "under", and "bed"?
Problem of Invariance: no one to one correspondence acoustics and phonetics
The Motor Theory of Speech Perception
Perception proceeds by reference to production
Place of articulation cued by the eyes
Manner of articulation cued by the ear
Transformation of acoustic signal to intended phonetic gesture is done in an automatic manner
A solution to the problem of invariance.
Evidence for the Motor theory:
What is the link between perception and production?
Teaching students foreign languages: Articulating new sounds help students hear them.
The McGurk Effect: Using articulatory knowledge during speech perception
speaker's lips say [ga], sounds says [ba], is perceived as [da]
The TRACE model of Speech Perception
(McClelland and Elman)three different interacting processing levels
• Feature level: nasality, acuteness, diffuseness etc.
• Phonemes: set of all phonemes in english
• words: set of words known
The role of activation in the TRACE model
• Activation of higher order units can also activate lower units
• inhibition occurs between units at the same level
Advantage of the model:
• Top down and bottom up processing
• Uses context to help figure out the word
• Coarticulation between words:
Foolish Capes
Christmas Capes
people hear a /t/ with /sh/ but a /k/ with /s/.
Articulatory differences influence perception of the initial phoneme of the second word
Elman and McClelland presented
FooliX capes
ChristmaX capes.
Found coarticulatory effects that people still heard the /k/ or the /t/.
Results similar to their model, indicates that there is top-down processing and the activation of word units influenced their interpretation of the t or k.
Perception of Written Language
Why did writing develop so late?
From a developmental point of view, recognizing printed word is the central problem of reading.
Questions:
• How long does it take to identify a word?
• Is identifying words effortful and the rest of the reading process automatic?
• Are words identified by accessing the sound and then the meaning?
• Are letters in words processed serially or are words processed as wholes?
• How much does context affect the process of word identification?
How long does it take to identify a word?
How would you do an experiment to identify a word?
Response-time method
Display a word: How long for it to be said?
400 msecs w/practice
Categorization task.
Is this an animal? "Dog", "Bacteria", "Rose", "Stone"
• For common instances, it takes about 700 msecs, but longer for "rose" and "bacteria"
Lexical Decision Task
• Is this a word?
• Simpler and faster than categorization
• Accessing some knowledge.
Brief Presentation Methods
• Give subjects only a short time to process the word
• Flash a word for only 60msec
• Masking
Estimates from Reading Text
• A college student reads about 300 words per minutes (5 words/sec=~200 msecs per word
• Problems with reading studies
Physiological Studies
• Evoked potentials: Well defined peaks in response to a word stimulus at 150-200msecs
Summary: How long does it take to identify a word?
Reaction Time studies: 400 msecs
Brief Presentation studies: > 60 msecs
Reading studies: 200 msecs
Physiological studies: 150- 200 msecs
What factors cause variation in identifying a word?
Coat vs. Cove:
Lexical decision task there is a difference of 100msecs
Naming times, the difference is only 30 msecs
The term "word identification" can mean a lot of things
Is word processing automatic?
What is taking up the time for doing the reading?
What do I mean by automatic?
• unaware of the process
• not under conscious control
• takes no processing capacity
Identifying the meanings of words is a rapid process for the skilled reader
Is processing words unconscious?
Priming study (Meyer & Schvaneveldt, 1971)
DOG #### CAT
FAN #### CAT
Lexical Decision 30 to 50 msecs faster for CAT if DOG is shown than if FAN is shown
Is processing words automatic?
Stroop effect. the name and meaning of words are processed even when the reader does not want to process the word.
Does word identification take processing capacity?
• Word identification takes only about 60 msecs of processing time,
• Reading takes about 200 msecs per word
• Identification is not a major part of the mental processing needed for reading
Are letters processed serially or are words processed as wholes?
Healy (1980) T-circling task
Evidence for reading units larger than letters
Number of missed ts ___ /40 = Percent errors on t
Number of missed thes _____/8 = Percent error in the
What percent of all the errors were made on the word the?
Number of missed thes / Number of missed ts
Why do we get the t-circling effect?
High-frequency words
Related effects:
• t circling in scrambled text
• proofreaders missing silent "e"
• circling "f" in "of"
The Word Superiority Effect
Do we process words serially letter-by-letter?
Cattell (1886)
Presented words or letters very briefly
Subjects were able to report words better than letters.
But, he
• did not control for guessing
• did not control for persistence of iconic memory
Reicher (1969)
• Pattern mask to control the stimulus presentation time.
• Changed the task to control for guessing
Subjects were better at picking out letters if they were in the context of words than if they were alone or if they were in a non-word
Is there a word superiority effect for psuedowords (mave)?
Are words then just visual templates?
Is there a detector for a particular pattern for each word?
dog, DOG, dOg, dog, dog, dog, dog, dog, dog dog, dog,
We can't have a template that would match each possible typeface that we would encounter.
We are not always processing words as "whole words"
A model of word recognition
McClelland & Rumelhart (1981) Interactive Activation model
• Detectors at three levels
• Processing occurs in parallel
• There is interaction among the levels, with activation flowing from detectors to other detectors
Eyemovement studies of reading.
Leisure reading: about 280 words per minute
What can our eye movements tell us about reading?
Eye Tracking device
• Presentation on a computer screen
• Position of eye is monitored
• Eye position is mapped on to position of text on screen
Fixations
• Rest periods
• 150-500 msecs
Saccades
• Rapid eye movements taking 20-35 msecs
• Ballistic movement
• Typically 7 to 9 character spaces with each saccade
• No visual information during saccade
Regressions
• Saccades that move backwards
• 10-15 percent of all saccades
Visual Acuity
• Foveal area subtends about 2 degrees of visual angle around fixation point
• Parafoveal area subtends 10 degrees of visual angle arounnd fixation point
Span of fixation
What is the size of the area from which a reader picks up information?
Moving Window technique (Rayner, 1975)
1) How large does a window have to be to allow normal
reading?
• 31 Characters (15 on each side)
2) What peripheral information is being used by the
reader?
Manipulate
• similarity of word shape
• word boundaries
• semantic relatedness
Measure fixation duration
Subjects show greater fixation durations with more visual and semantic inconsitencies
The captain granted the pass in the afternoon
The captain guarded the pass in the afternoon
The captain gnarbed the pass in the afternoon
The captain pmarved the pass in the afternoon
We use 7-12 character spaces for visual information (word shape, word boundaries)
We use 1-6 character spaces for semantic information
Conclusion: We do use information in our periphery, although we can use it more for visual information than for semantic informationAt what level are we processing a sentence?
The player caught the baseball
Syntactic, Semantic, Pragmatic levels
How do the levels interact?
Syntactic Processing
Parsing: Assigning linguistic categories to the surface structure of a sentence
The player caught the baseball
S -> NP+VP
NP -> det + (adj) + N
VP->V+NP
Parsing as problem solving or decision making.
How do we deal with syntactic ambiguity
They are flying planes
John drove down the street in the car
Yet, we don't make many problems parsing. Why?
Three possible ways of parsing
• Construct all logically possible syntactic structures
• Keep one structure, but if it doesn't work, go back and try another
• Hold information in short-term memory until there is enough evidence to rule out all but one structure
What are the problems with each approach?
Immediacy principle vs. Wait and see
Immediacy: When we see a word, we access, its meaning and fit it into the parse tree
Wait and See: We wait on interpreting a word until it is clear where it should fit
They are eating apples
However, immediacy can lead to errors in parsing
The florist sent the flowers . . .
. . .was very pleased
The florist who was sent the flowers was very pleased
Parsing strategies
What type of decisions are being made?
Late Closure strategy
The reader attaches new items to the phrase or clause currently being processed
Tom said that Bill had taken the cleaning out yesterday
1) Since Jay always jogs a mile seems like a short distance to him
2) Since Jay always jogs a mile this seems like a short distance to him
Minimal Attachment strategy
We prefer to attach new items into the phrase marker being constructed using the fewest syntactic nodes consistent with the rules of the language.
Ernie kissed Marcie and her sister ...
1) The girl knew the answer by heart.
2) The girl knew the answer was wrong.
We tend to take the answer as the direct object of the verb knew, but have to reinterpret in sentence 2 because the answer is part of a new clause
Evidence for Late Closure and Minimal Attachment strategies
1) Since Jay always jogs a mile seems like a short distance to him
2) Since Jay always jogs a mile this seems like a short distance to him
Word by word processing
Eyemovement studies
Longer fixations on disambiguating words, or the succeeding word, or regressions from those words.
Question: Do we deal with syntactic ambiguity in the same way as lexical ambiguity?
Do lexical (semantic) features interact with parsing?
Tyler and Marslen-Wilson (1977)
1) If you walk too near the runway, landing planes is...
2) If you walk too near the runway, landing planes are...
3) If you've been trained as a pilot, landing planes is ...
4) If you've been trained as a pilot, landing planes are...
Latencies are shorter for appropriate words rather than inappropriate words
Evidence for context helping to assign structural meaning to the sentence
Comprehending sentences in context
Parsing produces a syntactic structure. This permits a basis for determining the literal meaning of the sentence.
But what does a sentence mean?
Can you close the door?
We are fighting an uphill battle
He flew through his homework
Encyclopedias are goldmines
How do we use communicative context to understand sentences?
Conventions
Linguistic communication takes place within a context of shared assumptions about communication
Grice's maxims
conventions provide ground rules for successful conversation.
Joe: Are you going to the big dance tonight?
Sue: Didn't you hear that Billy Smith will be there?
Grices maxims:
All speakers adhere to the "cooperative principle" by following four maxims
Maxim of Quantity: Make your contribution as informative as is required, but not more so, for the current purpose of the exchange.
Maxim of Quality: Do not say what you belive to be false or that for which you lack adequate evidence.
Maxim of Relation: Say only what is relevant for the current purposes of the conversation
Maxim of Manner: Be brief, but avoid ambiguity and obscurity of expression.
Violating the conventions:
Steve: Jack is meeting a woman for dinner tonight.
Susan: Does his wife know?
Steve: Of course she does. The woman he is meeting is
his wife.
John went fishing. His boat sank.
What happens when you violate a maxim?
Speech acts.
Utterances that do not communcate much information but serve as an action.
I promise, ...... I'm sorry ... I'm happy for you
Types of speech acts:
Constatives: Expressing beliefs to form beliefs in the
listener
I deny that.
Directives: Expressing interest in the listener's future
action:
I asked you to do it.
Commissives: speaker is obligated to do something:
I promise to take out the garbage.
Acknowledgements: expresses feelings for the listener
I'm happy for you
Indirect speech acts
Communicative meaning does not correspond to the intended meaning
Question the ability of the listener
Can you close the window?
Question the listener's willingness
Will you close the window?
Indicate the reason why such an action needs to be done
It's getting windy in here
But ...a speech act has to meet certain conditions for it to be judged as appropriate
How do we comprehend speech acts?
Searle's stages:
1) listener extracts the literal meaning of the sentence
2) listener decides whether the literal meaning is what the speaker intended
3) Listener computes indrect meaning based on communicative conventions and direct speech act.
Testing Searle's model
Clark and Lucy (1975)
Used negation on the literal and the non-literal meaning
Subject's task matching pictures to sentences
1) Can you open the door?
postive in direct and indirect meaning
2) Must you open the door?
positive direct negative indirect
3) Why not open the door?
negative direct positive indirect
Positive requests had faster times than negative requests and requests in negative form to even longer.
Evidence for computing literal meaning of the sentence first.
Gibbs (1979)
Processing indirect speech acts in context
Must you open the window?
Mrs. Smith was watering her garden one afternoon. She saw the housepainter was pushing a window open. She didn't understand why he needed to have it open. A bit worried she went over and poiltey asked. "Must you open the window?"
One morning John felt too sick to go to school. The night before he and his friends got very drunk. Because of this he caught a bad cold. He was lying in bed when his mother stormed in. When she started to open the window John groaned, "Must you open the window?"
Then flash:
Need you open the window?
Do not open the window.
Indirect requests took longer in isolation but...
Subjects faster at identifying indirect requests in context than literal meaning.
Evidence that we don't compute a direct meaning if there is enough context.
Metaphor
We are fighting an uphill battle
He flew through his homework
He is drifting aimlessly
He is tied up at work
Arguments: = war
He attacked every week point in my argument
His criticisms were right on target.
He shot down all of my arguments.
If you use that strategy, he'll wipe you out.
Time:= money
You're wasting time
How do you spend your time?
I've invested a lot of time in her.
I can't spare the time.
How often do we use metaphors?
MacNeil/Lehrer News Hour: one metaphor every 25 words
What constitutes a metaphor?
Billboards are warts on the landscape
Topic/Tenor=Billboards
Vehicle= warts
Ground= Implied similarity between the tenor and vehicle
We use the tenor and vehicle to infer the ground
What is the tenor, vehicle and ground of these metaphors?
We hit a roadblock in our work
The thought slipped my mind like a squirrel behind a tree.
Oranges are the baseballs of the fruit lover.
He's as slow as a thumbtack.
Dogs are animals
Why do we use metaphors rather than literal statements?
Words are discrete while human experience is continuous.
Metaphors give us a way of describing information that would otherwise be hard to express
How do we comprehend metaphors?
Verbrugge and MCarrell (1977) Retention of information from metaphors.
Present metaphors, test recall of sentence by giving tenor, vehicle or ground.
Skyscrapers are the giraffes of the city
Give: "skyscrapers", or "giraffes", or "are very tall compared to surrounding things"
Results. Ground just as effective, as tenor and vehicle in recall.
We identify the underlying similarity of relations between tenor and vehicle.
Do we process the literal meaning of the metaphor?
With no context, metaphors take longer to process
With context, metaphors are processed just as fast as normal sentences
Do we sometimes not process the metaphorical meaning?
Subjects decided if a sentence is true.
All jobs are jails
Found that when metaphor was available in sentences that was false, response times were longer.
Indicates we can not ignore metaphors even when they are irrelevant to our task.
How do we process metaphors?
We retrievel lexical representation.
Determine what category they apply to and try to apply it
The pianist is a butcher
The surgeon is a butcher
Sermons are like sleeping pills
My husband is a teddy bear
Summary:
For both metaphors and speech acts, we comprehend the sentences directly, not interpretting the literal meaning.
Memory for sentences
What is remembered after exposure to a sentence?
Meaning vs. Surface Form
Strohner and Nelson 1974. Children aged two to three interpret sentences:
1) The cat chased the mouse.
2) The mouse was chased by the cat.
3) The mouse chased the cat.
4) The cat was chased by the mouse.
All interpret as "cat chased mouse"
By age 4 or 5 children interpret all four sentences correctly.... although sentences 3 and 4 take longer to process.
Does this happen for adults as well?
Task: Paraphrase the sentence
1) John dressed and had a bath.
2) John finished and wrote the article on the weekend.
3) Don't print that or I won't sue you.
Results in "If you print that I'll sue you
• 60 percent of people normalized perverse sentences
• If asked if there was any difference between what they said and the original sentence 53% said no difference.
• 47% of the time they justified it by saying that they knew what the sentence said, they just reworded it so that it was less clumsy.
A reality principle. "What is described in the discourse should be sensible"
Recognition memory for sentences
Fillenbaum (1966) Present long list of unrelated sentences
Present: 1) The window is not closed
Which did you see?
a) The window is closed
b) The window is not open
c) The window is open
Open and closed are "contradictories", does it work for "contraries"
The man was tall
The man was not short.
Maybe it is an effect of the experimental context
Wanner (1974) gave test on the instructions themselves
When you score the results, do nothing to your correct answer, but mark carefully those answers which are wrong.
Recognition test on
your correct vs. correct your
or
mark carefully vs. carefully mark
Subjects 100% recognize on your correct
Subjects 50% recognize on mark carefully.
People retain meaning and not surface form, even if they don't know if they are going to be tested on the sentence
Time course of retention of sentence information
How long do we hold surface and meaning?
Sachs (1967)
Played tape recorded passages.
Interrupt at various intervals.
Variables:
• Retention interval
• Type of sentence
Original
2 that changed wording but not meaning
1 changed wording and meaning
Results:
• If tested immediately: retained sentence wording and meaning
• If tested 40 syllables (12.5 secs) later: wording lost meaning stays
• If tested 80 syllables later: meaning still remains
Replicated for Visual presentations and American Sign Language
How do we retain information in the real world?
Keenan, MacWhinney and Mayhew (1977) retention of surface and meaning in natural conversations
Recorded researchers lunch discussions
Tested recognition memory
Interactional content : Jokes, Figures of speech, insults
Some utterances convey information to the listener, others convey attitude of the speaker toward the listener. These are high in interactional content
Results:
• High retention of both surface and meaning for sentences with high interactional content
• No retention of surface and low retention of meaning for sentences with low interactional content
Second test.
Presented sentences in a context free environment. No effect for interactional content.
Conclusion. It is the pragmatic factors of the conversational context that results in the retention.
The role of inferences in sentence memory.
Inference: A proposition, concept, drawn by the reader
Inferences are drawn through the process of elaboration
Example of inferences:
1) Last Christmas Emily went to a lot of parties.
This Christmas she got very drunk again.
2) Last Christmas Emily became absolutely smashed.
This Christmas she got very drunk again.
Second sentence takes longer to read in first passage.
Inferences and false recognition errors
What is from the text, what is an inference?
Bransford, Barclay and Franks (1972)
1) Three turtles rested beside a floating log, and a fish swam beneath them.
2) Three turtles rested on a floating log, and a fish swam beneath them.
Surprise memory test several minutes later.
Three turtles rested on a floating log, and a fish swam beneath it.
Three turtles rested beside a floating log, and a fish swam beneath it.
Subjects who had seen the on were more likely to falsely recognize the test sentence.
John was trying to fix the bird house. He was looking for the nail when his father came out to watch him and to help him do the work.
False recognition on:
John was using the hammer to fix the bird house when his father came out to watch him and to help him do the work.
Speed of processing of inferences
All the mugger wanted was to seal the woman's money. But when she screamed, he stabbed her with his weapon in an attempt to quiet her. He looked to see if anyone had seen him. He threw the knife into the bushes, took her money, and ran away.
People infer knife before getting to the last sentence,
reading time of last sentence the same if the second sentence had knife or weapon.
But, if the second sentence is:
But when she screamed, he assaulted her with his (knife/weapon) in an attempt to quiet her.
Last sentence takes longer to read when knife is not explicitly mentioned in the second sentence.
Conclusion: We draw elaborative inferences only when the context is clear
Inferences in advertising
• My doctor said Mylanta
• 4 out of 5 dentists recommend dentine for their patients who chew gum
• Breck Shampoo, now with vitamin E!
• My opponent voted against the child safety law act.
Harris (1977) are statements were true, false, or indeterminate
"Wouldn't it be great", asks the mother, "if you could make him coldproof? Well you can't. Nothing can do that (boy sneezes). But there is something that you can do that may help. Have him gargle with Gargoil Antiseptic. Gargoil can't promise to keep him cold-free, but it may help him fight off colds. During the cold-catching season, have him gargle twice a day with full strength Gargoil. Watch his diet, see he gets plenty of sleep, and there's a good chance he'll have fewer colds, milder colds this year."
Test : Gargling with Gargoil Antiseptic helps prevent colds.
Subjects can't distinguish between claims that are asserted vs. those that were implied.
Improving sensitivity to advertisements.
Harris (1977) some improvement in performance when subjects are explicitly warned about implied claims.
Bruno (1977): 20 minute training session, improved performance for both adults and school children.
How do we represent the gist in memory?
Propositions: Unit of meaning representing a phrase or clasue
a set of arguments joined by a predicate
The teacher taught the student
It was the teacher who taught the student
The student was taught by the teacher
taught(teacher, student)
The dog ate the large bone and then wagged his tail
p1: ate (dog, p2)
p2: (large, bone)
p3: and (p1, p4)
p4: wagged(his, tail)
How do we process information into propositions?
Comprehension and memory of connected discourse
Connected discourse: spoken and written: stories, class
lectures, manuals, textbooks ...
How is discourse organized and how does that affect our comprehension?
Discourse coherence and comprehension strategies
What is important for comprehension of connected discourse?
It is more than just understanding the meaning of each sentence
Carlos arranged to take golf lessons from the local professional. His dog, a cocker spaniel, was expecting pups again. Andrea had the car washed for the big wedding. She expected Carlos to help her move into her new apartment.
John bought a cake at the bake shop. The cake was chocolate with white frosting, and it read "Happy Birthday, Joan" in red letters. John was particularly pleased with the lettering. He brought it over to Greg's house, and together they worked on the rest of the details.
What makes the second paragraph easier to understand?
John bought a cake at the bake shop. The birthday card was signed by all of the employees. The party went on until after midnight.
Jeff went fishing. The boat sank, but he caught a big one.
Local/microstructure vs. global/macrostructure
Microstructure: relationship between individual sentences
Macrostructure: Relationship of sentences to our knowledge
Cohesion
Coherence is based on semantic relationships between sentences
"The range of possibilities that exist for linking something with what has gone before" (Halliday & Hassan, 1976)
Order these sentences:
A) He had eaten at McDonald's yesterday
B) Where should he eat?
C) Maybe Pizza Hut would deliver.
D) George needed to find some food.
E) However, he was tired of burgers.
What strategies did you use?
Categories of Cohesion
Reference, Substitution, Ellipsis, Conjuntion, Lexical
Anaphoric vs. cataphoric reference
Anaphoric reference: Current expresssion relates to an expression made earlier
cataphoric reference: Expression refers to future expression
This is how you do it. You let the herbs dry and then grind them up in a food processor.
I'm going to let you in on a secret. I hid the treasure on top of Deadman's bluff.
Strategies to establish coherence
Clark and Haviland: Distinction between given and new information
Given information: Information the author/speaker assumes the reader/listener already knows
New information: Information the reader/listener is assumed not to know.
Goal of giving information to someone is to provide both new information and old information.
It was Steve who robbed the bank.
It was the bank that Steve robbed.
The given/new strategy
1) Identify the given and new information in the current sentence
2) Find an antecedent in memory for the given information
3) Attach the new information to this spot in memory
Experimental approach: How long does it take to understand a sentence given what was said before?
Direct Matching Strategy
We got some beer out of the trunk.
The beer was warm.
Zak hopped into the waiting car and sped around the corner. He swerved to avoid the parked car and smashed into a building.
Zak hopped into the waiting car and sped around the corner. The old car lost a whell and smashed into the building.
Direct matches made on concepts not on words. (Overlap of arguments in the propositions)
p1: took (we, beer, p2)
p2: (from trunk)
p3: was(beer, warm)
Bridging inferences
We got the picnic supplies out of the trunk.
The beer was warm.
Last christmas John went to a lot of parties.
This Christmas he got very drunk again.
vs.
Last Christmas John became absolutely smashed.
This Christmas he got very drunk again.
Reinstating old information
Is the information in the background or the foreground?
I am trying to find a black dog.
He is short and has a dog tag on his neck that says Fred.
Yesterday that dog bit a little girl.
She was scared but she wasn't really hurt.
Yesterday a black dog bit a little girl
It got away and we are still trying to find it.
He is short and has a dog tag on his neck that says Fred.
She was scared but she wasn't really hurt.
Reinstatements: If information is no longer in the foreground, the reader must do a reinstatement.
Reinstatment requires retrieval of information from long term memory.
Comprehension time increases when subjects have to make reinstatements.
Coherence summary
If there is direct overlap we use: Direct match
Else
If there is some way to use semantic knowledge to find a relationship between current sentence and previous sentence by making a bridging inference
Else
Go back and search in memory for appropriate antecedentes by doing a reinstatement search
Memory for Discourse
How/What do we remember from reading a text?
A series of violent, bloody enounters between police and the Black Panther Party members punctuated the early summer days of 1969. Soon after, a group of Black students I teach at California State College, Los Angleles, who were members of the Panther Party, began to complain of continuous harassment by law enforcement officers. Among their many grievances, they complained about receiving so many traffic citations that some were in danger of losing their driving privileges. During one lengthy discussion, we realized that all of them drove automobiles with Panther Party signs glued to their bumpers. This is a report of a study that I undertook to assess the seriousness of their charges and to determine whether we were hearing the voice of paranoia or reality.
The van Dijk and Kintsch (1983) / Kintsch (1988) model of text comprehension
As text is read, it is processed into a set of propositions.
Propositions are connected to each other based on overlap of arguments in propositions (repetition rule)
Because of limitations of Short Term Memory, only a few propositions will be processed at a time.
More centrally connected propositions will receive the most activation
• These propositions will remain in short term memory longer
• Will tend to be transferred to long term memory and integrated with previous knowledge
• Will tend to be recalled the best
Key points:
Coherence created through the generation of connections between propositions
The levels effect Evidence for the hierarchical organization of memory.
Propositions at higher levels are recalled better
Limits on STM Text processing involves a limited capacity system with the goal of integrating new textual information with previous knowledge
The psychological validity of propositions
Reading times
Kintsch and Keenan created sentences that were about the same word length, but had different numbers of propositions
1) Cleopatra's downfall lay in her foolish trust in the fickle political figures of the Roman world.
2) Romulus, the legendary founder of Rome, took the women of the Sabine by force.
Increased processing time with increased numbers of propositions to process
Cued recall studies
Words from the same proposition are more effective recall cues than words from different propositions
The mausoleum that enshrined the czar overlooked the square.
overlooked will be a better recall cue for mausoleum than for czar in spite of its physical proximity
The representation of propositions in memory
McKoon & Ratcliff
Priming task after reading a short text
Early French settlements in North America were strung so thinly along the major waterways that land ownership was not a problem. The Frenchmen were fur traders, and, by necessity, the fur traders were nomads. Towns were few, forts and trading posts were many. Little wonder that the successful fur trader learned to live, act and think like an Indian. Circulation among the Indians was vital to the economic survival of the traders.
Use one proposition as a prime for a second proposition
Task: Is the second sentence true?
1) Circulation among the Indians was vital
The fur traders were nomads.
2) Land ownership was not a problem.
The fur traders were nomads
Sentences that were close in discourse stucture produce the greatest priming
Amount of priming is based on the number of intervening propositions, not on the number of intervening words
Inferences and Discourse representation
Inferences: generation of information that is not explicitly in the text
John got the picnic supplies out of the car.
The beer was warm.
Sara ate the chicken.
She was sick all night.
Under what conditions do we draw inferences?
McKoon & Ratcliff (1992)
We draw inferences only under two conditions:
1) When the inference is necessary to make the text locally coherent
2) When the information upon which the inference is based is easily activated
Stated explicitly in the text
Part of the reader's general knowledge
Representation of inferences
Are inferences added in to the reader's text representation in the same way as explicit information from the text?
Kintsch (1974)
Subjects read either:
1) A carelessly discarded burning cigarette started a fire.
The fire destroyed many acres of virgin forest.
2) A burning cigarette was carelessly discarded.
The fire destroyed many acres of virgin forest
Verification task: True or false A discarded cigarette started a fire
No difference between explicit and implicit propositions after 15 minutes.
Evidence for two representations:
1) short-term verbatim representation
Only contains explicit information mentioned in the text
2) long-term propositional representation
Both explicit and implicit information from the text stored in the same structure
Does this work for pictures as well?
Bagget (1975) picture stories
Same effect although longer delay (72 hrs) before implicit information is stored .
The role of the situation model
A representation of our general understanding of what the text refers to.
General background knowledge, Mental maps, Mental pictures
Information is easier to be learned when we can integrate that information into our situation model.
Example: Bransford and Johnson (1973)
If the balloons popped the sound wouldn't be able to carry since everything would be too far away from the correct floor. A closed window would also prevent the sound from carrying, since most buildings tend to be well insulated. Since the whole operation depends on a steady flow of electricity, a break in the middle of the wire would also cause problems. Of course, the fellow could shouut, but the human voice is not loud enough to carry that far. An additional problem is that a string could break on the instrument. Then there could be no accompaniment to the message. It is clear that the best situation would involve less distance. Then there would be fewer potential problems. With face to face contact, the least number of things could go wrong.
No picture: 26% recall of ideas from text,
very low rating of comprehensibility
With picture: 57% recall of ideas from text
middle rating of comprehensibility
Goal for comprehension process of reading: relate propositions currently being read to propositions that are already stored in memory
This requires working memory
Does reading ability depend on our working memory capacity?
Daneman and Carpenter complex reading span task
Subjects are read aloud a series of sentences and recall the final word of each sentence
• Number of sentences presented varied from 2 to 5
• Combining processing function with a memory/storage function
Different people have different reading spans
Significant correlation between reading span and
• comprehension test on questions
• pronoun reference tests
• Verbal SAT scores
Inferences and working memory capacity
Subjects provide Think aloud protocol while reading a passage
• High memory span subjects produce inferences at the end of passage
• Low memory span subjects produce inferences in the middle of the passage
What does this say about how we process text?
For low memory span subjects, difficulty of retaining information forces them to make inferences, even if they are wrong.
High memory span subjects can maintain multiple interpretations active in working memory and are able to make the correct interpretation when it is necessary.
Schemata and Discourse processing
How does the global structure of the discourse affect comprehension?
Genres: A type of discourse that has a characteristic structure
Class lectures, fairy tales, political speeches, News articles, Research articles, cop movies, romance novels
APA format:
Title page/Abstract
Introduction
Method
Subjects
Materials
Procedure
Results
Discussion
References
Appendices
Why does psychology use a single format?
Narrative Stories:
Introduction of Characters/Goal of main character
Introduction of Setting
Character runs into some obstacles
Character overcomes obstacles
Schemata and Discourse processing
Schema: a structure in memory the specifies the general or expected arrangement of a body of information
Restaurant schema
List 20 actions or events that occur commonly in eating at a restaurant.
Large agreement between subjects
All subjects mention:
Look at menu
order food
pay bill
Subjects also group the actions into similar components
Bartlett (1932) War of the Ghosts
Recall over time: How does the reader's schemata affect recall?
Process of "normalization" in recall: Stories that are incosistent with a reader's schemata are distorted in the direction of the schemata
Voss baseball study:
• Subjects listen to fictitious baseball game
• High knowledge subjects have better recall of events that are important for the game
• No difference between high and low knowledge subjects in recall information not related to the game's goal structure (weather, name of players)
Activation of an appropriate schema
If a text is written so that we can not activate the appropriate schema, recall will be poor.
The procedure is actually quite simple. First you arrange things into different groups. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities, that is the next step; otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too few things at once than too many. In the short run this may not seem important, but complications can easily arise. A mistake can be expensive as well. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one never can tell. After the procedure is completed, one arranges the materials into different groups again. Then they can be put into their appropriate places. Eventually they will be used once more, and the whole cycle will then have to be repeated. However, that is part of life.
Do we recall the details based on the activated schema?
Pichert and Anderson (1977)
Text describing the interior of the house.
Read the story from the perspective of:
buying a house
a burglar
Recall the story
now recall the story from the other perspective
After first recall, subjects recall more relevant information for their particular schema
After second recall, subjects able to recall previously unrecalled propositions from the second perspective
We are able to revive information by using a different schema
Conclusion:
• Schemata have a directive effect on comprehension
• Schemata help organize learning and affect recall of information
Story comprehension
How do we define a schema?
Story grammar: A schema that identifies the typical arrangement of events in a story
Mandler's rules for a story grammar
• Like phrase-structure rules
• Tree structure representation of stories
Application of story grammars to children's stories
All the animals at the zoo receive very good care. A zookeeper stays up each night to watch over them. One night the lions start making a lot of noise. One by one they started to roar and growl fiercely. The keeper was worred that they were sick or hurt. He wanted to see if something was wrong with them. He carried his bag of medicine into the lions' pen. The keeper fully checked over each one of the lions. He soon discovered that they were not sick at all. Thy were just trying to keep the other animals up. The zookeeper finally got them to go back to sleep. Soon things were quiet, and the zoo was calm again. Later on the elephants all got into a water fight. They used their trunks to toss water at each other. The keeper was mad because they were making a mess. He wanted them all to be quiet and peaceful again. He ran after the frisky elephants with a big stick. He shouted angrily at them to stop the water fight. The paid no attention to him and just ran faster. They squirted lots of water all over the poor man. They made an awful muddy mess in the elephant yard. The zookeeper worked for three days cleaning it up.
How do we represent and process episodes?
• Episodes tend to be recalled in an an all-or-none fashion (chunks)
• Length of episodes does not affect their recall
The time course of reading episodes
Haberlandt, Berian and Sandson (1980)
Sentence by sentence reading task, recording sentence reading time
Reading times were longer at the beginnings and ends of episodes
What processing is going on?
Begining of episodes
• Identifying new topic of discourse
• Forming expectations
End of episodes
• Summarize the gist of the episode
• Deciding what propositions to hold over
What is retained best from a story?
Attempts and outcomes are retained
Goals, reactions, emotions tend to be lost
Emphasis on what happenend more than what a character was thinking
Does the story grammar hold true for other cultures as well?
Mandler tested stories in Liberia
Wide range of literacy and education
Results show that recall patterns are similar despite different cultures and different degrees of experience with stories
Indicates that story grammar may be a universal schema
What do all these psychological studies say about teaching and learning?
• Strategies for better learning from text
• Strategies for improving texts
• Speedreading
What do you have to do to learn from a text?
Key issues:
• Learning surface structure vs. deep representation of text
• Identifying the important information in atext
• Connecting the discourse into a coherent body of information
• Activating the appropriate schema (if you have it) and connecting it to the information being processed
Active processing vs. passive processing
Relate what we read/hear to what we already know
Palinscar and Brown (1984)
Trained junior high students who were poor readers
• Trained to formulate questions that would be answered by the most important parts of the text
• Students learned to identify main themes in texts
• Post-test scores improved from 30% to 80%, no improvement for control group
• Long-term retention of these reading skills
Questioning the author (Beck)
• Why did the author write this?
• What is his/her point?
• How would I summarize this section?
• Where is he/she going with it?
Hilighting texts
Does using hilight pens help?
• Students read texts with or without hilight pens
• Students who did not hilight had higher comprehension
• It may help later search, but it did not foster active processing
• There are hilighting strategies that will help, though
We need to process information at a deep semantic level in order to retain it
Identifying the main points
What is remembered from a lecture?
Kintsch & Bates (1977)
• Students remembered recalled main points, but also recalled just as many details and jokes.
Problem for students: What are the main points of a lecture?
How do we identify the main points?
Explicit text signals can aid in comprehension of main points (Meyer, Brant & Bluth)
• Revised texts to improve signals
• Explicit: A problem of vital concern is the preventio of oil spills from supertankers
• Implicit: Prevention is needed of oil spills from supertankers
Explicit text improved comprehension of poor comprehenders
No difference in comprehension for good comprehenders
What about just leaving in the main points and taking the details out?
Reder & Anderson (1980)
Improved retention for condensed versions of texts
Discourse coherence
Goal in text comprehension: connect discourse into a well connected structure
Two ways of looking at coherence.
• Improving reader's strategies
• Improving the text's coherence
Improving reader's strategies
Reader's goal: create a well linked set of propositions
Look for coherence discourse markers at the global and local level
Local:
Sara went home. She was very tired
While this was the case, other people were not as pleased....
Global
A related point is...
In summary...
Topic headers
Draw inferences
• A cigarette was carelessly discarded. The fire caused a lot of damage.
• The plane was equiped for IFR flight. It should have managed to get through the storm.
Improving the text's coherence
Revising the coherence of the text at the local and global level
Study by McNamara, Kintsch, Songer & Kintsch
One text on heart disease revised into four versions improving local coherence and/or global coherence
Improving local coherence:
• Replace pronouns with noun phrases (replace it with heart)
• Adding sentence connectors (however, therefore, because, so that)
• Replacing words to increase argument overlap (repeat words across sentences)
Improving global coherence
• Add topic headers (Congenital Heart Disease, Acquired Heart Disease)
• Add macropropositions linking each paragraph to the rest of the text and to the topic. (There are many kinds of heart disease, some of which are present at birth and some of which are acquired later)
|
Low Local Coherence |
High Local Coherence |
|
|
Low Global Coherence |
||
|
High Global Coherence |
General result: Higher levels of coherence in text result in improved comprehension.
Why? Less cognitive strain on reader since the topic is unfamiliar
Second manipulation.
1/2 the students have high knowledge about heart disease
1/2 the students have low knowledge about heart disease
Results:
High knowledge students benefit most from low coherence texts
Low knowledge students benefit most from high coherence texts
Why? Force active processing for high knowledge students
Related applications
Britton & Gulgoz (1988)
Revised version of texts for Air Force textbooks
Revisions done based on propositional analysis to determine breakdown in coherence.
Results: Propositional analysis of a text can identify places where coherence can break down. Revising these places in the text can improve a reader's comprehension.
Building global structures
How does the current text fit in with your understanding of the topic?
Can you induce the author's schema?
Approaches:
• Try to summarize portions of the text
• Summarize each paragraph/section. How does it fit in with what you know?
Tailoring comprehension to how you will use the information
Try to encode/study the information in a similar manner to the way it will be used.
Mannes & Kintsch (1987)
Students provided with an outline of a topic that was either:
• consistent with the text
• inconsistent with the text
After reading the text, comprehension was evaluated
Students with the consistent outline performed better on memory for the text
Students with the inconsistent outline performed better on inference verification and problem solving
Why?
Inconsistent outline forced more active processing (deeper understanding)
Consistent outline permitted easier memorization of text but not deeper learning.
Review:
Strategies for improving your recall of a text.
• Active processing
• Identifying the main points
• Connecting the propositions in the discourse
• Building global structures
• Tailoring the comprehension activities to the test
Strategies for improving a text to make it easier to comprehend
• Revising local coherence
• Revising global coherence
• Tailoring the text to the reader's background knowledge/schema