1 00:00:43,160 --> 00:00:44,450 In Element 1, 2 00:00:44,550 --> 00:00:48,680 we have seen how to subtitle acoustic verbal elements. 3 00:00:49,240 --> 00:00:51,580 In LO1 of this Element, 4 00:00:51,840 --> 00:00:55,410 we have seen how to deal with the visual verbal elements 5 00:00:55,470 --> 00:00:56,860 of an audiovisual text. 6 00:00:57,620 --> 00:00:58,810 In this LO, 7 00:00:58,870 --> 00:01:02,770 we deal with the non-verbal elements of the speech to subtitle, 8 00:01:03,200 --> 00:01:05,570 be they acoustic or visual. 9 00:01:06,400 --> 00:01:09,230 This video lecture is aimed at letting you know 10 00:01:09,560 --> 00:01:14,780 how to identify which non-verbal elements need to be turned into verbal 11 00:01:15,110 --> 00:01:18,510 for each LTA trained working context. 12 00:01:19,000 --> 00:01:20,000 First of all, 13 00:01:20,060 --> 00:01:23,460 I will try to draw on the lessons learnt in LO1, 14 00:01:23,720 --> 00:01:25,770 meaning I will briefly recap 15 00:01:26,030 --> 00:01:30,820 the notions of text, context and semiotics of the audiovisual text. 16 00:01:31,510 --> 00:01:36,160 This will allow me to introduce the question of the communicative impact 17 00:01:36,230 --> 00:01:40,620 of each of the semiotic elements composing an audiovisual text 18 00:01:40,880 --> 00:01:44,910 on the overall communication of the audiovisual text itself. 19 00:01:45,570 --> 00:01:46,590 With this in mind, 20 00:01:46,720 --> 00:01:49,820 I will try to concentrate on non-verbal elements, 21 00:01:50,120 --> 00:01:51,770 both acoustic and visual, 22 00:01:52,200 --> 00:01:55,370 and how to deal with them in the subtitling process 23 00:01:55,670 --> 00:01:59,130 in case they play an important informative role. 24 00:02:02,660 --> 00:02:04,940 This is the agenda of this presentation. 25 00:02:11,650 --> 00:02:14,380 As said, in LO1 of this element 26 00:02:14,640 --> 00:02:16,790 we have seen the features of any text, 27 00:02:17,120 --> 00:02:19,030 which go beyond what is said. 28 00:02:19,690 --> 00:02:22,200 For each of the seven characteristics of a text, 29 00:02:22,530 --> 00:02:25,660 we have given concrete examples to be considered 30 00:02:25,730 --> 00:02:28,010 in real-time intralingual subtitling. 31 00:02:28,530 --> 00:02:33,020 In particular, we have seen the notions of cohesion, coherence, 32 00:02:33,290 --> 00:02:36,820 intentionality, acceptability, informativity, 33 00:02:37,150 --> 00:02:41,010 situationality and intertextuality. 34 00:02:41,630 --> 00:02:43,480 These are rarely considered 35 00:02:43,750 --> 00:02:47,010 if one thinks of respeaking as a mere repetition 36 00:02:47,340 --> 00:02:49,620 of what is said by a speaker. 37 00:02:51,140 --> 00:02:53,350 Then, we have moved to context. 38 00:02:53,840 --> 00:02:57,240 Respeakers not only have to focus on what a speaker says, 39 00:02:57,650 --> 00:02:59,300 but they also need to understand 40 00:02:59,530 --> 00:03:02,700 the relationship of what is said with the context 41 00:03:03,030 --> 00:03:05,240 if they reall want to provide subtitles 42 00:03:05,510 --> 00:03:08,050 that mean something to the target audience. 43 00:03:08,610 --> 00:03:09,610 Indeed, 44 00:03:09,660 --> 00:03:13,790 the meaning-making process of any speech event can only be understood 45 00:03:14,090 --> 00:03:16,590 in relation with its context. 46 00:03:17,190 --> 00:03:21,810 Furthermore, we have briefly analysed the three aspects of any context, 47 00:03:22,170 --> 00:03:26,160 that is its field, tenor, and mode. 48 00:03:27,090 --> 00:03:29,990 Concerning the semiotics of any speech event, 49 00:03:30,320 --> 00:03:33,590 we have finally seen that it communicates 50 00:03:33,720 --> 00:03:36,560 through the combination of many semiotic components. 51 00:03:37,050 --> 00:03:39,590 Be they TV programs, conferences, 52 00:03:39,660 --> 00:03:42,760 parliamentary sessions, meetings or school classes, 53 00:03:43,130 --> 00:03:46,620 they are all of either an acoustic or visual nature, 54 00:03:47,020 --> 00:03:49,360 and they are either verbal or non-verbal. 55 00:03:49,760 --> 00:03:53,450 In particular, we have analysed the different codes characterising 56 00:03:53,750 --> 00:03:57,120 the four semiotic macrocategories of a speech. 57 00:04:04,580 --> 00:04:06,590 Generally speaking, we tend to think 58 00:04:06,690 --> 00:04:11,800 that the verbal elements are more relevant than non-verbal elements 59 00:04:12,300 --> 00:04:16,880 and that acoustic elements are more relevant than visual elements. 60 00:04:17,380 --> 00:04:19,760 However, this is not always the case. 61 00:04:20,320 --> 00:04:24,670 The table in this slide shows the impact on communication 62 00:04:24,800 --> 00:04:29,720 of each semiotic element in the original version of an action movie 63 00:04:30,080 --> 00:04:31,770 and the corresponding impact 64 00:04:31,830 --> 00:04:35,690 of the same movie subtitled for the deaf and the hard of hearing. 65 00:04:36,390 --> 00:04:39,030 Of course, a movie is neither a conference speech, 66 00:04:39,320 --> 00:04:45,330 a parliamentary session, a news item, a school class nor a work meeting. 67 00:04:45,830 --> 00:04:47,900 However, this study shows 68 00:04:47,970 --> 00:04:52,290 that when the subtitler tries to translate the spoken into written, 69 00:04:52,820 --> 00:04:55,690 the attention of the audience is hijacked 70 00:04:55,890 --> 00:04:58,400 by the verbal component of subtitles. 71 00:04:59,440 --> 00:05:00,440 In particular, 72 00:05:00,580 --> 00:05:05,130 the acoustic verbal components and the visual verbal components together 73 00:05:05,460 --> 00:05:10,340 account for only 30% in the original version of a movie 74 00:05:10,740 --> 00:05:14,830 while they account for more than the double, 65%, 75 00:05:15,100 --> 00:05:17,970 in the same movie once it is subtitled. 76 00:05:18,590 --> 00:05:19,620 This happens 77 00:05:19,750 --> 00:05:25,820 because, a deaf or hard-of-hearing person can only rely on the visual channel 78 00:05:26,280 --> 00:05:29,060 to access the acoustic elements of a speech. 79 00:05:29,480 --> 00:05:30,610 In this context, 80 00:05:30,900 --> 00:05:36,650 the live intralingual subtitler turns almost all the verbal acoustic elements 81 00:05:36,810 --> 00:05:37,670 into subtitles, 82 00:05:38,260 --> 00:05:42,490 plus the non-verbal acoustic elements he or she deems essential 83 00:05:42,780 --> 00:05:45,560 to the understanding of the Source Text. 84 00:05:46,350 --> 00:05:50,600 Deciding whether to subtitle non-verbal acoustic elements or not 85 00:05:50,930 --> 00:05:52,750 is not always an easy task. 86 00:05:53,280 --> 00:05:56,310 This normally depends on how they contribute 87 00:05:56,540 --> 00:05:58,330 to the understanding of the event, 88 00:05:58,660 --> 00:06:00,040 or to its unfolding. 89 00:06:00,770 --> 00:06:03,340 If the respeaker decides to subtitle them, 90 00:06:03,640 --> 00:06:06,680 a caption describing the non-verbal acoustic occurrence 91 00:06:06,870 --> 00:06:08,360 is usually enough. 92 00:06:08,950 --> 00:06:13,180 However, the subtitler may not have captions ready at hand. 93 00:06:13,840 --> 00:06:17,740 In preparation of the event to subtitle, we suggest 94 00:06:17,830 --> 00:06:22,680 that the subtitler or the editor drafts a list of possible captions 95 00:06:23,010 --> 00:06:25,720 that could be used while subtitling the event. 96 00:06:26,440 --> 00:06:29,580 As you can see from the examples provided in this slide, 97 00:06:29,910 --> 00:06:33,140 we suggest using expressions that clearly state 98 00:06:33,570 --> 00:06:36,640 something that the audience can immediately understand 99 00:06:36,970 --> 00:06:39,480 like [voting starts] 100 00:06:39,840 --> 00:06:43,930 when the bell announcing the start of the voting sessions rings, 101 00:06:44,300 --> 00:06:49,870 instead of the ambiguous [bell] or [bell ringing]. 102 00:06:50,530 --> 00:06:56,310 The same is true for the caption saying [time for assessment is over] 103 00:06:56,870 --> 00:06:58,120 to tell a deaf student 104 00:06:58,190 --> 00:07:02,510 he or she has no more time to dedicate to writing his or her assessment, 105 00:07:03,110 --> 00:07:05,880 instead of [stop] 106 00:07:06,170 --> 00:07:09,340 or [teacher activated alarm clock]. 107 00:07:10,530 --> 00:07:15,650 Also, try to avoid general captions like [music] 108 00:07:16,040 --> 00:07:18,620 which means nothing to the eyes of the reader. 109 00:07:19,110 --> 00:07:24,320 Being more specific, like [Intersteno anthem being played], 110 00:07:24,690 --> 00:07:27,860 is for sure more useful and informative. 111 00:07:28,580 --> 00:07:31,850 Finally, try to be as exhaustive as possible 112 00:07:32,180 --> 00:07:36,800 and use more words than you may think are enough. 113 00:07:37,390 --> 00:07:40,590 Instead of [applauses], 114 00:07:40,890 --> 00:07:46,760 prefer [applauses from Lib Dems] if you are subtitling a political speech, 115 00:07:47,060 --> 00:07:51,550 as applauses may generate confusion and make the reader think 116 00:07:51,850 --> 00:07:54,950 the whole parliament approves what is said by the speaker. 117 00:07:55,710 --> 00:07:58,480 What is important to understand here is that, 118 00:07:58,710 --> 00:08:02,900 depending on the setting, such captions are more or less needed. 119 00:08:03,360 --> 00:08:07,320 For example, during conferences their use is limited 120 00:08:07,620 --> 00:08:11,280 to some recurrent things like [applauses from audience] 121 00:08:11,380 --> 00:08:13,860 or [Wim laughs]. 122 00:08:14,580 --> 00:08:16,100 During a meeting, 123 00:08:16,370 --> 00:08:20,500 the person receiving personalised access may be more interested knowing 124 00:08:20,560 --> 00:08:21,910 what happens around them. 125 00:08:22,870 --> 00:08:25,210 In the news, it is very common to see 126 00:08:25,280 --> 00:08:28,210 captions translating non-verbal acoustic events, 127 00:08:28,440 --> 00:08:34,420 like [explosion], [ambulance syren], [shots], and similar. 128 00:08:36,460 --> 00:08:39,240 In the case of non-verbal visual elements, 129 00:08:39,660 --> 00:08:43,330 the decision of the respeaker to translate them in the subtitles 130 00:08:43,390 --> 00:08:46,200 also depends on the way subtitles appear on screen. 131 00:08:46,990 --> 00:08:51,120 If they appear as two liners bottom of the screen showing the speaker, 132 00:08:51,870 --> 00:08:53,190 the task is easy, 133 00:08:53,360 --> 00:08:56,920 as the viewer can watch the screen more or less simultaneously 134 00:08:56,990 --> 00:08:58,180 and read the subtitles. 135 00:08:58,970 --> 00:08:59,990 However, 136 00:09:00,060 --> 00:09:04,080 in a conference you may have the speaker projecting a presentation on a screen, 137 00:09:04,450 --> 00:09:06,560 and the subtitles appearing on another one. 138 00:09:07,150 --> 00:09:11,770 In this case the viewer’s experience needs to be facilitated somehow. 139 00:09:12,430 --> 00:09:16,000 Similarly to what happens in dealing with visual verbal elements, 140 00:09:16,430 --> 00:09:21,050 in this case, you may opt for either a momentary transitional caption 141 00:09:21,340 --> 00:09:23,620 telling the audience to watch the other screen 142 00:09:24,080 --> 00:09:28,410 or you can add some words verbalising what images mean. 143 00:09:29,730 --> 00:09:34,280 In the case of a speaker showing a picture and speaking about the picture, 144 00:09:34,610 --> 00:09:36,890 you can either use a transitional caption 145 00:09:37,150 --> 00:09:42,000 like that in the first column which says [watch slide] 146 00:09:42,360 --> 00:09:45,370 or you can verbalise the reference to the picture 147 00:09:45,660 --> 00:09:49,530 and add something like «as you can see from the slide» 148 00:09:50,090 --> 00:09:51,270 to the speaker’s output. 149 00:09:51,900 --> 00:09:54,410 Similarly, when a speaker shows a chart 150 00:09:54,740 --> 00:09:58,530 where several lines indicate how given sectors of a company perform, 151 00:09:58,830 --> 00:10:04,640 it may be useful to inform the audience about what happens in the other screen. 152 00:10:05,070 --> 00:10:09,660 Again, you can either use a caption saying [watch the red line] 153 00:10:10,020 --> 00:10:13,620 or add «the red line indicating a 1% growth» 154 00:10:14,040 --> 00:10:15,200 to the speaker’s output, 155 00:10:15,500 --> 00:10:18,900 if he or she does not mention the piece of data. 156 00:10:20,310 --> 00:10:23,320 Another example may be that of a speaker using a map 157 00:10:23,650 --> 00:10:27,840 to show the position of a city like Milan, without telling it. 158 00:10:28,760 --> 00:10:31,720 The respeaker can either make use of a transitional caption 159 00:10:31,800 --> 00:10:34,210 saying [watch map] 160 00:10:34,570 --> 00:10:35,960 or verbalise the event 161 00:10:36,390 --> 00:10:40,020 by adding «in the north of Italy» to the speaker’s output. 162 00:10:40,710 --> 00:10:45,200 A last case of visual non-verbal element is a new speaker speaking. 163 00:10:45,760 --> 00:10:49,120 This is very common when subtitling classes and meetings, 164 00:10:49,490 --> 00:10:53,120 but also in other contexts like parliamentary sessions, 165 00:10:53,310 --> 00:10:55,860 the news, or the Q&A after a conference. 166 00:10:56,610 --> 00:10:59,250 There are several options to signal a new speaker. 167 00:10:59,750 --> 00:11:02,220 One solution is that of using a name tag, 168 00:11:02,650 --> 00:11:06,840 which is the name of the speaker followed by a colon or a dash 169 00:11:07,110 --> 00:11:10,470 or the name of the speaker into parentheses or brackets. 170 00:11:10,940 --> 00:11:12,780 If you don’t know the name of the speaker 171 00:11:12,980 --> 00:11:16,940 you can simply go to a new line and start the new subtitle with a dash. 172 00:11:17,570 --> 00:11:19,650 In some cases, especially on TV, 173 00:11:20,040 --> 00:11:23,380 turns can be signalled by changing the colour of the text. 174 00:11:24,070 --> 00:11:25,790 Similarly to what happens 175 00:11:25,820 --> 00:11:28,690 with the previous cases of acoustic non-verbal elements 176 00:11:29,020 --> 00:11:30,800 and visual verbal elements, 177 00:11:31,100 --> 00:11:34,960 we suggest you draft a list of these strategies 178 00:11:35,220 --> 00:11:39,580 before the event so as to be ready and use them when needed. 179 00:11:43,840 --> 00:11:46,020 In this video lecture we have summed up 180 00:11:46,210 --> 00:11:50,070 some of the lessons learnt in LO1 of Element 2 181 00:11:50,500 --> 00:11:52,550 and then we have tried to provide 182 00:11:52,580 --> 00:11:57,370 an overview of the strategies to be used in different working contexts 183 00:11:57,760 --> 00:12:00,800 to turn non-verbal elements into subtitles. 184 00:12:01,330 --> 00:12:04,500 In particular, we have recalled the notions of text, 185 00:12:04,590 --> 00:12:08,090 context and semiotics of an audiovisual text, 186 00:12:08,520 --> 00:12:13,110 to then move to the impact non-verbal elements have in an event 187 00:12:13,470 --> 00:12:17,960 and how this impact changes when the same event is subtitled. 188 00:12:18,550 --> 00:12:22,710 Finally, we have seen some of the strategies used to verbalise 189 00:12:22,980 --> 00:12:26,010 both acoustic and visual non-verbal elements. 190 00:12:26,610 --> 00:12:30,730 In the homework session, you will be asked to try and provide solutions 191 00:12:30,990 --> 00:12:34,620 for each situation where non-verbal elements are essential 192 00:12:35,050 --> 00:12:37,260 for the good understanding of a speech.