This is the abstract for a paper presented at the Future of Theory in Education Conference in Stirling, June 7-9 2012, and it is now the abstract for a paper forthcoming in MIT's Leonardo journal in 2014, which explores the way learning and knowledge is based in 'rich synaesthesia' and cross-modal perception, patterns, and thought, in more detail. (A draft of the full - 2014 - paper for can be accessed here )

SYNAESTHESIA: from cross-modal to modality-free learning and knowledge.
Roy Williams, University of Portsmouth, UK
Simone Gumtau, University of Portsmouth, UK
Jenny Mackness, Independent Education Consultant, UK

Theory and research on interactive media and learning has started to move away from cognitivism and the Cartesian separation of body and mind, to an embodied view of the world, and a view of perception and action as enactive-perception. This leads to an approach based on investigating all the senses together, the interaction between them and, increasingly, cross-modality rather than just multi-modality – in perception, action, interaction, and making sense of the world. In short: synaesthesic experience, learning and action. This paper outlines the key aspects of this emerging research field, and ways in which these theories and frameworks have been applied to learning and to learning design: for instance, Montessori mathematics, and interactive spaces for children on the autistic spectrum. This paper explores the way in which learning theory seems to be moving on from notions such as constructivist, contextual, inquiry- or problem-based, learner-centred, or multi-media learning, to a far more integrated notion of ‘whole body’ engagement in the world.

Keywords: synaesthesia, embodied learning, enactive perception, enactive meaning.

And ....

In the process of writing and presenting this paper at the Future of Theory in Education conference in Stirling (see above), and revising it for publication in Leonardo, we changed how we think about embodiment, perception, affordances, and syn/aesthetics, as well as new mashed-up concepts like 'synaesthesic scaffolding' and even 'synaesthesic orchestration'.

To start with, we now see synaesthesia and embodiment as two sides of the same coin. Synaesthesia is, for us, necessarily embodied, sensory, cross-modal, in a way that first of all 'passes through' the cortex, without "perception being subverted by language" (a quote from Costall in the paper), and one could take that one step further, and say that not only perception, but also cross-modal engagement can also occur without being 'subverted by language'. It takes Gibson's idea of 'direct perception' and transforms it into 'direct engagement' in a literally syn-aesthetic sense.

We are still wrestling with this, but it struck me over the last few weeks, and in the discussions at Stirling, that what was previously wrong with 'our' thinking (speaking for myself, firstly) was that I thought of perception (and synaesthesia) as something that fed into a neat hierarchical development: first the synaesthesic, followed by the textual and textually-mediated semiotic, which then becomes more and more abstract, in a rather linear, teleological fashion in which you 'ascend' from the messy world of perception to the 'pure' worlds of text and abstraction.

Now I think of it quite differently - I think of it as a dialectic, starting with a process of poly-modal perception, which almost immediately merges into cross-modal perception, and perceptive-action (speech, dance, gesture - are of them all highly cross-modal), which are always, already, engaging and fascinating on thier own terms. Then I think of language (and speech / 'parole') as a kind of hybrid synaesthetic/textual bridge between the 'pure use value' of cross-modal engagement, and the 'pure exchange value' of written texts. And texts are like a stripped-down, culinary-'reduced' version of engagement, highly stylised - many written languages differ substantially from the way they are spoken by most of their 'natural' speakers (see for example written Arabic v. Egyptian spoken Arabic, or written English, which is based on a highly stylised RP or 'received pronunciation' - also known as 'BBC' English, which is only spoken by a small minority of people in England).

And then comes the surprise, because the next step is that there are peculiar examples of the abstract, below (Brancusi's Sleeping Muse, and Henry Moors's Atom Piece) which seem to be very abstract, but in fact they return to the intuitive, to what we can call the intuitive-aesthetic, as Brancusi himself pointed out: "There are idiots who define my work as abstract; yet what they call abstract is what is most realistic. What is real is not the appearance, but the idea, the essence of things.”

external image Brancusi's+Head.JPG

Brancusi: Seeping Muse
When it works best, the intuitive–aesthetic is paradoxically both intuitive and constructed, without too much in-your-face mediation. It's why seeing Brancusi's Head, or Moore'e Atom Piece - i.e. ‘seeing’, in the ontological, being-there, walking-around-it-sense - is essential for an appreciation of what the pieces are about. Your engagement with these artefacts 'feeds off' the multi-sensory engagement, especially proprioceptively, as with each movement of your head/ body you get to 'see' something else - other things 'come to mind'.

external image Atom+Piece.JPG

Henry Moore: Atom Piece, study for Nuclear Power

The same thing happens for me with some performances. To engage with a dance performance by moving about would be even better, and I wonder what would happen if the audience could walk around the dancers (the Royal Albert Hall comes to mind). This would (start to) transform the performance/audience into a collaborative dance - which happens in live performance to some extent (as a dancer you have to engage with the audience, and they have to imaginatively engage with you), but the audience is sort of 's/trapped in their chairs'.

I think this kind of intuitive-aesthetic is very possible, and would in principle be much more possible if the audience could literally move about an 'arena in the round'.

Perhaps ... a sunken stage for wheel-chair performers, surrounded by a raised platform, on which the audience sits in ... wheelchairs, why not? ... that they can use to move around the platform. (Perhaps a trite example, but it has possibilities,no?)

This gives us a whole new spectrum of affordances:
  • intuitive
  • textual
  • intuitive-aesthetic

As well as:
  • synaesthetic scaffolding, and
  • synaesthetic orchestration.

[Cross-posted to: ]