Find subgoals in attested assembly paths
An earlier study [1] found that people offload working memory to the physical world when solving a virtual block-copy task. Essentially, the participants in this study solved the task one block at a time by looking at the reference model to decide which block to add next, then looking again to decide where it should be placed on the copy.
It seems plausible that successful assembly plans might also make use of sub-goals---i.e. when trying to copy a reference model, you focus on correctly building a sub-part of it before moving on to the next one. This could make the copy task easier by reducing the search space when checking which block to select from the basket, or where to place it on the copy model.
In our study we've observed that our participants overwhelmingly create states corresponding to correct vertical layers of the reference model, although their strategies for completing the layers vary. Could these layer states be evidence of sub-goal planning? Below are some ideas for ways we could test this hypothesis.
- Look for pauses between layer 1 and layer 2
- After a layer has been built, look for eye movements that do not align with actions annotated during the builds where the participant would have been observing what is on the table and what to do next before launching into the build for the next layer. We could have a period of time that is designated as a significant pause and can look at which models/layers seem to cause participants to hesitate the most.
- Look for changepoints in stability across paths
- When generating the paths, print log of large variations in stability by outputting assemblystats.isStable. My idea is that we can set a threshold for what quantifies a large enough changepoint in stability between two states, and we can log/output the configuration of the state before and after the change to see if there is a common pattern between these changes onto a log, and then can potentially run analysis on the text of that log in another script to find patterns.
- Look for changepoints in number of looks to the model across paths---do they coincide with stability changepoints?
- Using the eyetracking pipeline will allow us to look at the differences between model look backs across different builds within a particular type of construction to compute if there are significant variations in model look backs in people's approaches to builds.
- By using the significant stability changepoints computed from idea 2 and using the eyetracker data processing pipeline to align where participants were looking at the given state computed in stage 2 (stage 3 of the pipeline does this conversion) and then by looking at the number of times they look at the model approaching this stability changepoint (using the event timestamp converted to an eyetracking timestamp) we can see if they appear to coincide.
- Compare gaze sequences between people with identical assembly paths
- Generate a script to output identical paths for each model, run the eyetracking pipeline on these paths/videoIds and compare their output.
[1] "Deictic Codes for the Embodiment of Cognition" Dana H. Ballard, Mary M. Hayhoe, Polly K. Pook, and Rajesh P. N. Rao. Behavior and Brain Science 1997.