Multimodality: The Five Semiotic Systems

Multimodality: the five semiotic systems (bull & Anstey, 2018)

TSLB3433 Multiple Literacies

learning objectives

in this lecture, you will:

• Elaborate the key concepts in multimodality

• Explain the codes and conventions of the five semiotic systems:

  • Linguistic

  • Visual

  • Audio

  • Gestural

  • Spatial

• Analyse different multimodal texts

TSLB3433 Multiple Literacies

Key concepts in multimodality

​1. Multimodal texts as dynamic design

• continue to change

• have different meanings among cultural groups​

TSLB3433 | Multiple Literacies

Key Concepts

Key concepts in multimodality

​2. Reconceptualising reading and writing as consuming and producing

• ​the agency of the consumer and producer

• awareness of how the interests and influences (i.e. literacy identity) of the consumer or producer mediate the text and the meaning-making process​

• literacy identity includes:

  • prior experience with texts

  • knowledge about texts

  • social and cultural knowledge and experiences

  • technological knowledge and experiences​

TSLB3433 | Multiple Literacies

Key Concepts

Open Ended

Why should we reconceptualise reading and writing as consuming and producing in multimodal texts?

Key concepts in multimodality

​3. Designing text

• ​is about meaning making

• recursive

• non-sequential

• ​interactive

• can adhere to known conventions or be innovative

• involves synthesising​

• transformation = the result of the process of design​

TSLB3433 | Multiple Literacies

Key Concepts

Key concepts in multimodality

​4. Importance of metacognition and metalanguage

• ​Metacognition:

  • Knowing that (​a range of metacognitive strategies)

  • Knowing how (how to use the strategies)

  • Knowing when (when best to use the strategies)

  • Knowing why (why the strategy selected is the best one to use)

• Metalanguage to talk about:

  • The codes and conventions of the semiotic systems

  • ​The thinking processes involved in consumption and production

  • The structure of text

  • The technologies associated with producing and consuming text

TSLB3433 | Multiple Literacies

Key Concepts

5 Concepts about multimodal texts

​1. Multimodal texts convey meaning through a combination of meaning making elements that draw upon several semiotic systems.

TSLB3433 | Multiple Literacies

Key Concepts

Multiple Select

Which semiotic systems are involved in producing this text?

• Linguistic

• Visual

• Audio

• Gestural

• Spatial

Multiple Select

Which semiotic systems are involved in producing this text?

• Linguistic

• Visual

• Audio

• Gestural

• Spatial

5 Concepts about multimodal texts

​1. Multimodal texts convey meaning through a combination of meaning making elements that draw upon several semiotic systems.

TSLB3433 | Multiple Literacies

Key Concepts

5 Concepts about multimodal texts

​2. Multimodal texts draw upon and cross the boundaries of the Arts (the disciplines of Art, Media Studies and the Performance Arts - Music, Drama and Dance), their knowledge, understandings and processes.

​

3.​ The role of language in multimodal text varies; it is not always the dominant semiotic system present.

​

TSLB3433 | Multiple Literacies

Key Concepts

5 Concepts about multimodal texts

​4​. In a multimodal text, meaning is distributed across all elements (parts) and each element has a role in contributing to the overall (whole) meaning of the text.

• The roles might be:

  • complementary

  • reinforcing

  • hierarchical

  • oppositional (used less often)

​

TSLB3433 | Multiple Literacies

Key Concepts

5 Concepts about multimodal texts

​5. The codes and conventions of the semiotic systems in a multimodal text can be used to identify the relative importance of meaning making elements and provide coherence throughout the text.​

TSLB3433 | Multiple Literacies

Key Concepts

Tutorial tasks

in groups of five, complete the following:

1. Explain the codes and conventions of one of the semiotic systems. Refer to pages 98-130 in Elaborating Multiliteracies through Multimodal Texts: Changing Classroom Practices and Developing Teacher Pedagogies by Bull and Anstey (2019).

2. ​Select any multimodal text and analyse it in details.

3. Upload your presentation slides in GC and present it in the next class.

TSLB3433 Multiple Literacies