Vyxal, 56 49 46 bytes

(ṫ~c[€ɖ‡ṘNf&›|p;ȧ:(ṫ€:ḣt↔h:ḃ∇v€ÞfḣṘJf)W∑¥[d¥+N

Try it Online!

First we extract all cross-caps. This is done by looking at the last symbol. If it occurs in the same orientation twice, the cross-cap is extracted by taking a word \$PaQa\$ and transforming it into \$PQ^{-1}\$, incrementing the register each time it happens. Otherwise the word is rotated to the left. This is repeated as many times as the number of symbols in the input.

(ṫ~c[€ɖ‡ṘNf&›|p;
(              ; # run the following once for each symbol in the input:
                                               [1, 2, 3, 1, -2, 3]
 ṫ               #   tail extract              [1, 2, 3, 1, -2], 3
  ~c             #   contains? without popping [1, 2, 3, 1, -2], 3, 1
    [            #   if:
     €           #     split on                [[1, 2], [1, -2]]
      ɖ‡--       #     apply the following to the second item:
        ṘN       #       reverse and negate
                                               [[1, 2], [2, -1]]
          f      #     flatten                 [1, 2, 2, -1]
           &›    #     increment register
             |   #   else:
              p  #     prepend

Now we are left with a word of an orientable surface so we can forget which symbols are inverted.

Now we extract handles and remove adjacent inverses. This is done by looking at the last symbol. We write the word as \$PaQa\$. If there is a common symbol in \$P\$ and \$Q\$ there exists a handle \$RbSaTbUa\$ so we transform the word into \$RUTS\$, leaving a 1 on the stack each time it happens. If there are no common symbols in \$P\$ and \$Q\$ then \$a\$ will eventually become adjacent with itself so we can delete it, leaving a 0 on the stack each time it happens. This is repeated as many times as the number symbols left in the word after extracting cross-caps.

ȧ:(ṫ€:ḣt↔h:ḃ∇v€ÞfḣṘJf)
ȧ                      # absolute value
 :(                  ) # run the following once for each symbol in the word:
   ṫ                   #   tail extract
    €                  #   split on
     :                 #   duplicate
      ḣ                #   head extract
       t               #   tail
        ↔              #   intersection
         h             #   head
          :            #   duplicate
           ḃ           #   boolify
            ∇          #   shift (leave it on the stack for later)
             v€        #   vectorise split on
               Þf      #   flatten by one level
                 ḣ     #   head extract
                  Ṙ    #   reverse
                   Jf  #   join and flatten

All that's left is to combine the number of cross-caps extracted and number of handles extracted.

W∑¥[d¥+N
W        # wrap stack in a list
 ∑       # sum
  ¥[     # if register is non-zero
    d    #   double
     ¥+  #   add register
       N #   negate

Curry (PAKCS), 159 bytes

f[]=0
f(p++a:b:q)|a+b==0=f$p++q
f(p++a:q++a:r)=g$f$p++reverse[-x|x<-q]++r
f(p++a:q++b:r++c:s++d:t)=(a+c)!(b+d)$f$p++s++r++q++t
g x=min(-2*x)x-1
(0!0)x|x>=0=x+1

Attempt This Online!