Using the Facial Action Coding System (FACS): a point of view from a FACS Coder

INTRODUCTION

I prefer to leave burden and honour to the authors (Paul Ekman) for a possible technical explanation about what is the FACS – Facial Actions Coding System, about the extraordinary scientific applications  have been made since its creation and new frontiers and future applications, and about who can be an awesome mentor in FACS (Erika Rosenberg) and an excellent trainer that could prepare people to take the FACS Test Exam.

Every detail on how to use the FACS will be exhaustively explained itself in the Manual, which is and will always represent for a FACS Coder an indispensable guide, to be consulted every times codify facial expressions is necessary.

As “old school” FACS Coder Certificate (when I was learning FACS it was still in a CD-ROM format, I studied it non-stop for 6 months with many effort and difficulty, doing every day the exercises at the end of every Chapters of the Manual before felt me comfortable and ready for the exam) that is lucky in apply this extraordinary method in its profession as Consultant, I will try to give only some practical advice to the new FACS Coders who wants to start coding photos and videos of facial expressions.

Dr. Luca Marzari

 

What does the face express?

According to Ekman, the face has four types of signals, able to convey informations:

  • static: more or less permanent aspects of the face. Examples are: skin pigmentation (color); the shape of the face; the bone structure; cartilages; fat deposits; the size, shape and position of the features (for example: eyebrows, eyes, nose, mouth).
  • slow variation: changes in appearance that gradually occur over time. Examples are: permanent wrinkles; changes in muscle tone; the alterations of the grain and pigmentation of the skin (in old age).
  • rapid: movements of the facial muscles, which produce transient changes in the appearance of the face. Examples are: temporary wrinkles and changes in shape and position of the features (these changes are fast on the face: they last seconds or fractions of a second).
  • artificial: use of accessories or other, which disguise the usual appearance of the face. Examples are: glasses; a long beard; the shape of the hair (fringe or other); the use of cosmetics (to change skin color, cover wrinkles, etc.); the use of botulinum, which gives a “stretched” and youthful appearance to the face, but partially paralyzes nerves and muscles, affecting the possibility of performing facial expressions.

The facial signals that are important for FACS Coder are the rapid signals.

Before to notice them we must discover the baseline of the neutral faceof the subjects we want to observe, in order to keep under control the variable “personal physiognomic peculiarity of a subject” = for remove the influences of possible pre-existing static signals and artificial signals. After that, we could focus our attention in codify only the rapid signals (emotional or communicative facial expressions).

Let’s take an example, in order to better clarify this extremely important point.

A video of an experimental subject is delivered to a FACS Coder; the subjects face in the foreground has been videotaped.

The subject had watched videos that triggered him strong emotions (sadness, anger, fear and happiness).

If the FACS Coder wants to avoid errors or false positives, he must first notes every pre-existing static signal:

  • the FACS Coder note that the shape of the subject’s mouth is particular, as the angles are turned slightly downwards, by default;

Click for extend the picture

  • the FACS Coder note in addition that the subject has bags visible under the eyes, in the region of the lower eyelid;

Click for extend the picture

  • CASE LIMIT: the subject has a thick beard covering his face, making it difficult to notice the slight movements of the cheeks or lips;
  • CASE LIMIT: the subject has a fringe, which partially covers the forehead;
  • CASE LIMIT: the subject is lit too much or too less, so the video is of poor quality;
  • CASE LIMIT: the subject has a frame of the black glasses, which partially covers the shape of the eyes and eyebrows;
  • CASE LIMIT: the subject has no make-up, but the shots immortalize him with a filter that makes it almost impossible to notice wrinkles or shapes, and alters their features.

If the FACS Coder disregard all these elements, she/he risk to make a lot of mistakes:

  • could misunderstand the info and decodify (EMFACS), from the beginning of the video, a presence of sadness that may not exist in the observed subject, misinterpreting the angles of the mouth facing downwards;
  • could decodify anger or cognitive load, exchanging natural bags under the eyes of the subject as activations or tension of the lower eyelid;
  • if the FACS Coder, in a lazy way, does not focus himself on the upper face part (eyes, eyebrows and forehead), assuming that nothing can be seen because of the fringe and glasses, it will probably risks to lose a lot of useful information.
  • however, asking to short-sighted subject to remove his/her glasses, could create to him/ to her extreme problems in watching the video stimuli: for sure he/she will put down and reunite his/her eyebrows and / or tighten the lower eyelids, because of the effort in try to see better. It could be wrongly interpreted as a facial expression of cognitive loading (thinking, remembering, taking time to respond), or as a controlled anger. But of course it is not.

A good observer codifies and annotates each element, taking care of the onset, the peak and the offset of the movement in order to distinguish what is stably present from the beginning (baseline) from these momentary elements, and from this difference the FACS Coder will continue to analyze the next movements of the muscles of the face.

An another baseline should be considered, watching the entirety of the video, in order to collect all the non-verbal behaviors who are not useful or that create negatively influence to the interpretation (meaningless grimaces or possible tics, great postural changes, itching, considering whether the subject is talking during the video or if he chews a chewing-gum, etc.). Reduce the noise, before to do errors.

Check every movements of cheeks and mouth before to codify mouth's AUs

Brief description of the Facial Action Coding System (FACS)

Paul Ekman and Wally Friesen (1978) devoted themselves to the drafting of the first atlas of the human face, the Facial Action Coding System (FACS), with a systematic description in words, photographs and films of how to measure facial movements in anatomical terms, breaking them down into single movement units (AU – Action Unit).

Ekman and Friesen have calculated 44 AUs (Action Units) that consider variations in facial expressions, and 14 AUs that interpret the changes in the direction of the gaze and in the orientation of the head. The combinations obtainable, by the combined effect of several Action Units activated at the same time in the face, cover the totality of all the possible facial configurations that a human being can assume (more than 10,000 in total).

A FACS Certified Coder can interpret almost all facial expressions, breaking them down into specific units of action and subdividing them into their temporal phases.

Useful illustrations that will facilitate the comprehension and the learning (performed in high intensity – D or E level – with the only goal of facilitating the viewing of the AUs in action), are included in Table 1.

FACS is a research tool useful for measuring any facial expression that a human being can achieve. The individual Action Units constitute the essence of the whole “atlas”; in Table 1 some examples are now provided with: nomenclature (name of the AU), descriptive incipit (movement and description), images of the separate activation of individual Action Units, compared with images of the neutral face of the subject.

Important: Table 1 was created with only one goal of being a brief and helpfull memorandum of the Facial Action Coding System AUs (like pag. 526 of the FACS Manual), in order to cover some possible (human and normal) temporary “lack of memory” of the FACS Coders, and could be consulted in case of doubt during a facial expressions videos analysis. This online tool is NOT the FACS Manual and could NOT substitute it in any occasion: only the FACS – Facial Actions Coding System Manual and Investigators Guide are the most complete and reliable tools for deeply study the amazing world of human Facial Expressions.

More info:

Table 1

Action Unit

AU 0

Movement / Description

None

Facial Muscle involved

None

AU 0 (Neutral Face)

Action Unit

AU 1

Movement / Description

Inner Brow Raiser

Facial Muscle involved

Frontalis, pars medialis

From AU 0 to AU 1

Action Unit

AU 2

Movement / Description

Outer Brow Raiser

Facial Muscle involved

Frontalis, pars lateralis

From AU 0 to AU 2

Action Unit

AU 4

Movement / Description

Brow lowerer

Facial Muscle involved

Frontalis, pars medialis

From AU 0 to AU 4

Action Unit

AU 5

Movement / Description

Upper Lid Raiser

Facial Muscle involved

Levator palpebrae superioris

From AU 0 to AU 5

Action Unit

AU 6

Movement / Description

Cheek Raiser

Facial Muscle involved

Orbicularis oculi, pars orbitalis

From AU 0 to AU 6

Action Unit

AU 7

Movement / Description

Lid Tightener

Facial Muscle involved

Orbicularis oculi, pars palpebralis

From AU 0 to AU 7

Action Unit

AU 8 (+AU 25 + AU 26)

Movement / Description

Lips toward

Facial Muscle involved

Orbicularis oris

From AU 0 to AUs 8+25+26

Action Unit

AU 9

Movement / Description

Nose Wrinkler

Facial Muscle involved

Levator labii superioris alaquae nasi

From AU 0 to AU 9

Action Unit

AU 10

Movement / Description

Upper Lip Raiser

Facial Muscle involved

Depressor Glabellae, Depressor Supercilli, Currugator

From AU 0 to AU 10

Action Unit

AU 11

Movement / Description

Nasolabial Deepener

Facial Muscle involved

Zygomatic Minor

From AU 0 to AU 11

Action Unit

AU 12

Movement / Description

Lip Corner Puller (bilateral and unilateral)

Facial Muscle involved

Zygomatic Major

From AU 0 to AU 12 & U12

Action Unit

AU 13

Movement / Description

Cheek Puffer

Facial Muscle involved

Levator anguli oris (Caninus)

From AU 0 to AU 13

Action Unit

AU 14

Movement / Description

Dimpler (bilateral and unilateral)

Facial Muscle involved

Buccinator

From AU 0 to AU 14 & U14

Action Unit

AU 15

Movement / Description

Lip Corner Depressor

Facial Muscle involved

Depressor anguli oris (Triangularis)

From AU 0 to AU 15

Action Unit

AU 16 (+ AU 25)

Movement / Description

Lower Lip Depressor

Facial Muscle involved

Depressor labii inferioris

From AU 0 to AUs 16+25

Action Unit

AU 17

Movement / Description

Chin Raiser

Facial Muscle involved

Mentalis

From AU 0 to AU 17

Action Unit

AU 18

Movement / Description

Lip Puckerer

Facial Muscle involved

Incisivii labii superioris and Incisivii labii inferioris

From AU 0 to AU 18

Action Descriptor

AD 19

Movement / Description

Tongue show

Facial Muscle involved

From AU 0 to AD 19

Action Unit

AU 20

Movement / Description

Lip stretcher

Facial Muscle involved

Risorius

From AU 0 to AU 20

Action Unit

AU 21

Movement / Description

Neck tighten

Facial Muscle involved

From AU 0 to AU 21

Action Unit

AU 22 (+ AU 25)

Movement / Description

Lip Funneler

Facial Muscle involved

Orbicularis oris

From AU 0 to AUs 22+25

Action Unit

AU 23

Movement / Description

Lip Tightener

Facial Muscle involved

Orbicularis oris

From AU 0 to AU 23

Action Unit

AU 24

Movement / Description

Lip Pressor

Facial Muscle involved

Orbicularis oris

From AU 0 to AU 24

Action Unit

AU 25

Movement / Description

Lips part

Facial Muscle involved

Depressor Labii, Mentalis (relaxation of it), Orbicularis Oris

From AU 0 to AU 25

Action Unit

AU 26 (variants)

Movement / Description

Jaw Drop

Facial Muscle involved

Masetter; Temporal and Internal Pterygoid (relaxation of them)

From AU 0 to AU 26 (AUs 17+24 involved)

From AU 0 to AUs 25+26

Action Unit

AU 27 (variants)

Movement / Description

Mouth Stretch

Facial Muscle involved

Pterygoids, Digastric

From AU 0 to AU 27 (AUs 17+24 involved)

From AU 0 to AUs 25+27

Action Unit

AU 28 (+ AU 26)

Movement / Description

Lip Suck

Facial Muscle involved

Orbicularis oris

From AU 0 to AU 28

Action Descriptor

AD 29

Movement / Description

Jaw thrust

Facial Muscle involved

From AU 0 to AD 29

Action Descriptor

AD 30

Movement / Description

Jaw sideways

Facial Muscle involved

From AU 0 to AD 30

Action Unit

AU 31

Movement / Description

Jaw clench

Facial Muscle involved

From AU 0 to AU 31

Action Descriptor

AD 32

Movement / Description

Bite lip

Facial Muscle involved

From AU 0 to AD 32

Action Descriptor

AD 33

Movement / Description

Blow

Facial Muscle involved

From AU 0 to AD 33

Action Descriptor

AD 34

Movement / Description

Puff

Facial Muscle involved

From AU 0 to AD 34

Action Descriptor

AD 35

Movement / Description

Cheek suck

Facial Muscle involved

From AU 0 to AD 35

Action Descriptor

AD 36

Movement / Description

Tongue bulge

Facial Muscle involved

From AU 0 to AD 36

Action Descriptor

AD 37

Movement / Description

Lip wipe

Facial Muscle involved

From AU 0 to AD 37

Action Unit

AU 38

Movement / Description

Nostril dilate

Facial Muscle involved

Nasalis; Pars Alaris

From AU 0 to AU 38

Action Unit

AU 39

Movement / Description

Nostril compress

Facial Muscle involved

Nasalis; Pars transversa and depressor septi nasi

From AU 0 to AU 39

Action Descriptor

AD 40

Movement / Description

Sniff

Facial Muscle involved

From AU 0 to AD 40

Action Unit

AU 43

Movement / Description

Eyes Closed

Facial Muscle involved

Levator Palpebrae Superioris (relaxation of it)

From AU 0 to AU 43

Action Unit

AU 45

Movement / Description

Blink

Facial Muscle involved

Levator palpebrae superioris (relaxation of it); Orbicularis oculi pars palpebralis

From AU 0 to AU 45

Action Unit

AU 46

Movement / Description

Wink

Facial Muscle involved

Levator palpebrae superioris; Orbicularis oculi, pars palpebralis

From AU 0 to AU 46

Action Descriptor

AD 50

Movement / Description

Talk

Facial Muscle involved

From AU 0 to AD 50

Head and Eye

51

Movement / Description

Head turn left

Facial Muscle involved

From AU 0 to 51

Head and Eye

52

Movement / Description

Head turn right

Facial Muscle involved

From AU 0 to 52

Head and Eye

53

Movement / Description

Head up

Facial Muscle involved

From AU 0 to 53

Head and Eye

54

Movement / Description

Head down

Facial Muscle involved

From AU 0 to 54

Head and Eye

55

Movement / Description

Head tilt left

Facial Muscle involved

From AU 0 to 55

Head and Eye

56

Movement / Description

Head tilt right

Facial Muscle involved

From AU 0 to 56

Head and Eye

57

Movement / Description

Head forward

Facial Muscle involved

From AU 0 to 57

Head and Eye

58

Movement / Description

Head back

Facial Muscle involved

From AU 0 to 58

Head and Eye

M59

Movement / Description

Head shake up and down

Facial Muscle involved

From AU 0 to M59

Head and Eye

M60

Movement / Description

Head shake side to side

Facial Muscle involved

From AU 0 to M60

Head and Eye

61

Movement / Description

Eyes turn left

Facial Muscle involved

From AU 0 to 61

Head and Eye

62

Movement / Description

Eyes turn right

Facial Muscle involved

From AU 0 to 62

Head and Eye

63

Movement / Description

Eyes up

Facial Muscle involved

From AU 0 to 63

Head and Eye

64

Movement / Description

Eyes down

Facial Muscle involved

From AU 0 to 64

Head and Eye

65

Movement / Description

Walleye

Facial Muscle involved

From AU 0 to 65

Head and Eye

66

Movement / Description

Cross-eye

Facial Muscle involved

From AU 0 to 66

Head and Eye

M68

Movement / Description

Upward rolling of eyes

Facial Muscle involved

From AU 0 to M68

Head and Eye

M69

Movement / Description

Eyes positioned to look at other person; Head and/or eyes look at other person

Facial Muscle involved

From AU 0 to M69

Visibility codes

70

Movement / Description

Brows and forehead not visible

Facial Muscle involved

From AU 0 to 70

Visibility codes

71

Movement / Description

Eyes not visible

Facial Muscle involved

From AU 0 to 71

Visibility codes

72

Movement / Description

Lower face not visible

Facial Muscle involved

From AU 0 to 72

Visibility codes

73

Movement / Description

Entire face not visible

Facial Muscle involved

From AU 0 to 73

Visibility codes

74

Movement / Description

Unsociable

Facial Muscle involved

From AU 0 to 74

Action Descriptor

AD 80

Movement / Description

Swallow

Facial Muscle involved

From AU 0 to AD 80

Action Descriptor

AD 81

Movement / Description

Chewing

Facial Muscle involved

From AU 0 to AD 81

Action Descriptor

AD 82

Movement / Description

Shoulder shrug

Facial Muscle involved

From AU 0 to AD 82

Head and Eye

M83

Movement / Description

Head upward and to the side

Facial Muscle involved

From AU 0 to M83

Action Descriptor

AD 84

Movement / Description

Head shake back and forth

Facial Muscle involved

From AU 0 to AD 84

Action Descriptor

AD 85

Movement / Description

Head nod up and down

Facial Muscle involved

From AU 0 to AD 85

Unclear Codes

AD 91

Movement / Description

Flash

Facial Muscle involved

From AU 0 to AD 91

Unclear Codes

AD 92

Movement / Description

Partial Flash

Facial Muscle involved

From AU 0 to AD 92

Wikipedia Adds

97*

Movement / Description

Shiver/tremble

Facial Muscle involved

From AU 0 to AD 97

Wikipedia Adds

98*

Movement / Description

Fast up-down look

Facial Muscle involved

From AU 0 to AD 98

The individual Action Units, if activated at any given time, are called Combination of AUs. Some examples (among the over 10,000 possible combinations) are shown in the Examples of Facial Expressions (ENG).