Lip Sync Animation

Part 1

by Didik Wijaya

In this tutorial you will learn to make a character talks. I suggest that you read previous tutorial Creating Facial Expression In 3ds Max. The Morpher technique in previous tutorial is applied here also. In this tutorial I will use only Morpher to animate the character. This will make the animation simpler.

1. First, we are going to need a character model. You can download file here. You can open file cartoonbear_lipsync_start.max. This is a cartoon bear character. This model is not the same with character in Creating Facial Expression In 3ds Max tutorial. Here, I removed bones in ears. Also I add a tounge. I also modify the face so it has neutral looks. This is a condition when the character doesn't speak

Basically, when you want to make a character speak. You need to animate lips, teeth, tounge and jaw. In this tutorial I don't animate the jaw.

2. Because we are going to use Morpher, we need to create several morpher targets with different mouth shape. Each of these mouth shapes is usually referred to as a phoneme. A phoneme is the unique shape a mouth takes to make a specific sound. For example, to speak "food", your mouth tends to pucker into a small circle. The word "Animation" could be described phonetically as "a-nee-may-shun" or more like a breakdown as "a-n-ee-m-ay-sh-u-n".

Each morpher target is created from cloning the original head object, then modify its mouth shape. Delete Skin modifier in cloned object if it is ruined. Use unique name for each cloned head. Read Creating Facial Expression In 3ds Max tutorial for further explanation on creating morpher target.

To create animation, I usually create 13 different mouth shapes. Here they are:

Phoneme	How it sounds	Example words
A	Ape	Apple, blAde, Ape
I	Eye	Eye, Under
O	Oat	Open, OAt, Over
E	Eat	swEEt, EAt, feet
W OO Q	Oo	WOOd, drEW, QUIet, fOOd
M B P	Mm	Map, Bang, toP
C K G	Sea	Carry, looK
CH SH J	Church	CHerry, SHout, Jump
N D T L	End	Name, Dove, abouT, faLL
TH	Thin	teeTH, forTH, THat
S Z	Zee	Snow, Zoo, Sneer
R	Earth	Roll, dooR, wondER
F V	Fish	Fine, LoVe

3. Beside making phoneme targets, you also need to create different facial expressions. Later you can combine phoneme with facial expression. Usually there are 6 major expressions: Fear, Anger, Joy, Sad, Disgust, and Surprise.

Sample of finished 3ds max file up to this point is available in downloaded file. (cartoonbear_lipsync_part1.max). In the next tutorial you will learn how to load morpher target and animate them.

part 2

1. Select original head object. Apply Morpher modifier. Place this modifier under Skin modifier. In Modify tab, click "Load Multiple Targets" button. In opened window, load all objects.

Try to adjust spinner value to check the animation When finished, make sure all spinner value is in 0.0 The character is ready to be animated.

Sample of finished file up to this point is available (cartoonbear_lipsync_part2.max)

2. Next, load sound. You can use file "oh-yeah-everything-is-fine.wav" available in download file. Open Curve Editor. Highlight Sound. Right click and choose Properties. Click Choose Sound and load the wav file. When finished, close Curve Editor.

You will see sound in waveform like image below. It spreads from frame 0 to 50. I wrote phonetic sounds and frames assosiated with it as guidance.

Tips: You may think that we need to create phonemes for all letters, but actually we don't need to. We just need to create several phonemes, just to "imitate" the sound like shown in image below.

3. Are you ready to animate? Turn on Auto Key.

I listed keyframes you need to create in table below. For example phoneme O is created from frame 2 to 7, with its peak value at frame 4. Here's how to do it. Move slider to frame 2, in Modify tab you can see spinner value in phoneme O is at 0.0. Click spinner in phoneme O down. The value still 0.0, but this will create a keyframe (shown in read color in spinner).

Move slider to frame 4, adjust spinner at 100. Then, move slider at frame 7 an set spinner value=0.

Repeat this process untill all phonemes listed below are animated. When finished, turn off Auto Key. Your character now SPEAKS!

Phoneme	Start (0)	Peak (100)	End(0)
O	2	4	7
I	7	13	17
F	24	27	30
TH	28	30	33
E	34	35	37
F	38	40	43
I	40	42	46

4. Video below shows sample of lip sync animation. The character will say "Oh yeah, everything is fine." I also add a slight JOY expression. Btw, when you play video below, you may see unexpected animation at the end. I don't know what caused this, maybe from youtube. Original video file can be found here.