Uiua, 9 bytes

/+⊂∩♭∩⧈-⟜⍉

Try it: Uiua pad

APL(NARS), 19 chars

{+/∊-2-/¨a b←⍉a←⊃⍵}

This should apply -2-/ to the rows both of matrix a build on ⍵ and its traspose b. Than flatty the array and sum the numbers +/∊

  {+/∊-2-/¨a b←⍉a←⊃⍵}(8 7 1)(4 1 3)(5 5 5)
¯9
  {+/∊-2-/¨a b←⍉a←⊃⍵}(1 2 3)(4 5 6)(7 8 9)
24
  {+/∊-2-/¨a b←⍉a←⊃⍵}(1 2)(1 2)
2
  

Japt -x, 11 10 9 8 bytes

cUÕ cÈän

Try it

cUÕ cÈän     :Implicit input of 2D array U
c            :Concatenate
 UÕ          :  U transposed
    c        :Flat map
     È       :Through the following function
      ä      :  Consecutive pairs reduced by
       n     :    Inverse subtraction
             :Implicit output of sum of resulting array

Vyxal d, 5 bytes

:∩"v¯

Try it online, or verify all test cases.

Explanation:

:      # Duplicate
 ∩     # Transpose
  "    # Pair
   v¯  # Deltas of each
       # Flatten and sum with the d flag

Factor + math.unicode, 48 bytes

[ dup flip [ [ differences Σ ] map Σ ] bi@ + ]

Try it online!

Explanation:

It's a quotation (anonymous function) that takes a sequence of sequences (matrix) from the data stack as input and leaves a number (sum) on the data stack as output. Assuming { { 1 2 3 } { 4 5 6 } } is on the data stack when this quotation is called...

• dup Duplicate an object.

  Stack: { { 1 2 3 } { 4 5 6 } } { { 1 2 3 } { 4 5 6 } }

• flip Transpose a matrix.

  Stack: { { 1 2 3 } { 4 5 6 } } { { 1 4 } { 2 5 } { 3 6 } }

• [ [ differences Σ ] map Σ ] Push a quotation for bi@ to use later.

  Stack: { { 1 2 3 } { 4 5 6 } } { { 1 4 } { 2 5 } { 3 6 } } [ [ differences Σ ] map Σ ]

• bi@ Apply a quotation to two objects. (Inside the quotation now...)

  Stack: { { 1 2 3 } { 4 5 6 } }

• [ differences Σ ] Push a quotation for map to use later.

  Stack: { { 1 2 3 } { 4 5 6 } } [ differences Σ ]

• map Take a sequence and a quotation and apply the quotation to each element in the sequence, collecting the results into a new sequence of the same length. (Inside the quotation now...)

  Stack: { 1 2 3 }

• differences Calculate the first-order differences of a number sequence.

  Stack: { 1 1 }

• Σ Sum the elements of a sequence.

  Stack: 2

• Now map applies the quotation to the next row and the result is also 2.

  Stack: { 2 2 }

• Σ Sum the elements of a sequence.

  Stack: 4

• Thus, the first result of bi@ is 4. Now bi@ repeats the process for its second input (the transposed matrix) and ends up with 9.

  Stack: 4 9

• + Add.

  Stack: 13

Brachylog, 22 16 bytes

⟨≡{s₂ᶠc+ᵐ-}ᵐ\⟩+ṅ

Try it online!

(-6 bytes inspired by @Kroppeb's suggestions.)

?⟨≡{s₂ᶠc+ᵐ-}ᵐ\⟩+ṅ.       Full code (? and . are implicit input and output)
?⟨≡{       }ᵐ\⟩          Apply this on both the input and its transpose:
    s₂ᶠ                  Get pairs of successive rows, [[row1, row2], [row2, row3], ...]
       c                 Flatten that: [row1, row2, row2, row3, row3, row4, ...]
        +ᵐ               Sum the elements within each row [sum1, sum2, sum2, sum3, ...]
          -              Get the difference between even-indexed elements (starting index 0)
                         and odd-indexed elements, i.e. sum1+sum2+sum3+... - (sum2+sum3+sum4+...)
                         This gets the negative of the usual difference i.e. a-b instead of b-a
                         for each pair of rows
               +         Add the results for the input and its transpose
                ṅ        Negate that to get the sign correct
                 .       That is the output

Brachylog, 13 bytes

orignally based of @sundar's design

⟨≡⟨t-h⟩ᵐ²\⟩c+ 

Explanation

⟨≡      \⟩          #   Take the original matrix and it's transpose 
      ᵐ             #       and execute the following on both
       ²            #           map for each row (this is now a double map "ᵐ²")
  ⟨t h⟩             #               take head and tail
   -                #               and subtract them from each other (sum of deltas in a row)
         c+         #       and add all the values 
                    #           (we have two arrays of arrays so we concat them and sum them)

the ⟨⟩ are messing up formatting, sorry

Try it online!

Pyt, 11 bytes

Đ⊤ʁ-⇹ʁ-áƑƩ~

Explanation:

          Implicit input (as a matrix)
Đ         Duplicate the matrix
⊤         Transpose the matrix
ʁ-        Get row deltas of transposed matrix
⇹         Swap top two elements on the stack
ʁ-        Get row deltas of original matrix
á         Push the stack into an array
Ƒ         Flatten the array
Ʃ         Sum the array
~         Flip the sign (because the deltas are negative, as subtraction was performed to obtain them)
          Implicit output

APL (Dyalog Classic), 12 bytes

(+/⊢⌿,⊣/)⌽-⊖

Try it online!

MY, 9 bytes

ωΔω⍉Δ ḟΣ↵

Try it online!

Since I cannot ping Dennis in chat to pull MY (due to a suspension), this will currently not work. (Δ previously didn't vecify when subtracting) Thanks to whomever got Dennis to pull MY!

How?

• ωΔ, increments of the first command line argument
• ω⍉Δ, increments of the transpose of the first command line argument
• , in a single list
• ḟ, flatten
• Σ, sum
• ↵, output

Haskell, 60 bytes

e=[]:e
z=zipWith
f s=sum$(z(-)=<<tail)=<<(s++foldr(z(:))e s)

Try it online! Uses the shorter transpose I found a while ago.

Explanation

e is an infinite list of empty lists and used for the transposing. z is a shorthand for the zipWith function, because it is used twice.

f s=                                        -- input s is a list of lists
                            foldr(z(:))e s  -- transpose s
                         s++                -- append the result to the original list s
                     =<<(                 ) -- map the following function over the list and concatenate the results
        (z(-)=<<tail)                       -- compute the delta of each list by element-wise subtracting its tail
    sum$                                    -- compute the sum of the resulting list

APL, 18 15 bytes

{-+/∊2-/¨⍵(⍉⍵)}

Try it online!

CJam, 19 bytes

0q~_z+2few:::-:+:+-

Input is a list of lists of numbers. Try it online!

Explanation

0       e# Push 0
q~      e# Evaluated input. 
_       e# Duplicate
z       e# Zip (transpose)
+       e# Concatenate. This gives a lists of lists of numbers, where the
        e# inner lists are the original rows and the columns
2few    e# Replace each inner list of numbers by a list of overlapping
        e# slices of size 2. We not have three-level list nesting
:::-    e# Compute difference for each of those size-two slices. We now
        e# have the deltas for each row and column
:+      e# Concatenate all second-level lists (de-nest one level)
:+      e# Sum all values
-       e# Subtract from 0, to change sign. Implicitly display

J, 14 bytes

+/@,&({:-{.)|:

Try it online!

Explanation

+/@,&({:-{.)|:  Input: matrix M
            |:  Transpose
     (     )    Operate on M and M'
      {:          Tail
        -         Minus
         {.       Head
   ,&           Join
+/@             Reduce by addition

MATL, 7 bytes

dG!dhss

Try it online!

Explanation:

Suppose the input is

[8 7 1; 4 1 3; 5 5 5]

d        % Difference between rows of input
         % Stack:
         % [-4 -6  2; 1  4  2]
 G       % Grab the input again. Stack:
         % [-4 -6  2; 1  4  2]
         % [8 7 1; 4 1 3; 5 5 5]]
  !      % Transpose the bottom element. Stack:
         % [-4 -6  2; 1  4  2]
         % [8 4 5; 7 1 5; 1 3 5]
   d     % Difference between rows. Stack:
         % [-4 -6  2; 1  4  2]
         % [-1 -3  0; -6  2  0]
    h    % Concatenate horizontally. Stack:
         % [-4 -6  2 -1 -3  0; 1  4  2 -6  2  0]
     ss  % Sum each column, then sum all column sums. Stack:
         % -9

Octave, 33 bytes

@(x)sum([diff(x)(:);diff(x')(:)])

Try it online!

Explanation:

This is an anonymous function taking x as input. It takes the difference between all columns, and concatenates it with the difference between the columns of the transposed of x. It then sums this vector along the second dimension.

R, 34 bytes

function(m)sum(diff(m),diff(t(m)))

Try it online!

Anonymous function. Originally, I used apply(m,1,diff) to get the rowwise diffs (and 2 instead of 1 for columns) but looking at Stewie Griffin's answer I tried it with just diff and it worked.

Husk, 7 bytes

ΣṁẊ-S+T

Try it online!

-1 thanks to Mr. Xcoder taking my focus away from the S and ¤ and towards the m (which should've been ṁ).
-1 thanks to Zgarb abusing S.

Explanation:

ΣṁẊ-S+T 3-function composition
    S   (x -> y -> z) (f) -> (x -> y) (g) -> x (x) (implicit): f x g x
     +    f: [x] (x) -> [x] (y) -> [x]: concatenate two lists
      T   g: [[x]] (x) -> [[x]]: transpose x
 ṁ      (x -> [y]) (f) -> [x] (x) -> [y]: map f on x and concatenate
  Ẋ       f: (x -> y -> z) (f) -> [x] (x) -> [z]: map f on splat overlapping pairs of x
   -        f: TNum (x) -> TNum (y) -> TNum: y - x
Σ       [TNum] (x) -> TNum: sum x

JavaScript (ES6), 68 67 bytes

m=>m.map(r=>s+=[...l=r].pop()-r[0],s=0)|m[0].map(v=>s+=l.pop()-v)|s

Formatted and commented

m =>                              // given a matrix m
  m.map(r =>                      // for each row r of m
    s += [...l = r].pop() - r[0], //   add to s: last value of r - first value of r
    s = 0                         //   starting with s = 0
  ) |                             //
  m[0].map(v =>                   // for each value v in the first row of m:
    s += l.pop() - v              //   add to s: last value of last row of m - v
  ) |                             //
  s                               // return s

Because the minimum size of the input matrix is 2x2, m.map(...)|m[0].map(...) is guaranteed to be coerced to 0. That's why it's safe to return the final result with |s.

Test cases

let f =

m=>m.map(r=>s+=[...l=r].pop()-r[0],s=0)|m[0].map(v=>s+=l.pop()-v)|s

console.log(f( [[1, 2], [1, 2]]                                   )) // => 2
console.log(f( [[8, 7, 1], [4, 1, 3], [5, 5, 5]]                  )) // => -9
console.log(f( [[1, 2, 3], [4, 5, 6], [7, 8, 9]]                  )) // => 24
console.log(f( [[9, 9, 9, 9, 9], [9, 9, 9, 9, 9]]                 )) // => 0
console.log(f( [[1, 3, 14], [56, 89, 20], [99, 99, 99]]           )) // => 256
console.log(f( [[1, 2, 3, 4], [4, 5, 6, 7], [7, 1, 8, 2]]         )) // => 9
console.log(f( [[13, 19, 478], [0, 12, 4], [45, 3, 6], [1, 2, 3]] )) // => -72

Mathematica, 45 bytes

Tr@Flatten[Differences/@#&/@{#,Transpose@#}]&

Input

[{{13, 19, 478}, {0, 12, 4}, {45, 3, 6}, {1, 2, 3}}]

Python 2, 42 bytes

lambda m:sum(r[-1]-r[0]for r in m+zip(*m))

An unnamed function taking a list of lists, m, and returning the resulting number.

Try it online!

How?

The sum of the deltas of a list is the last element minus the first, everything else just cancels:
(r[n]-r[n-1])+(r[n-1]-r[n-2])+...+(r[2]-r[1]) = r[n]-r[1]

The zip(*m) uses unpacking (*) of m to pass the rows of m as separate arguments to zip (interleave) and hence transposes the matrix. In python 2 this yields a list (of tuples, but that's fine), so we can add (concatenate) it to (with) m, step through all our rows and columns, r, perform the above trick for each and just add up the results (sum(...)).

Pyth, 7 bytes

ss.+M+C

Try it here.

My first ever answer in a golfing language! Thanks to @EriktheOutgolfer for -1 byte!

Explanation

ss.+M+C    ~ This is a full program with implicit input (used twice, in fact)

      C    ~ Matrix transpose. Push all the columns;
     +     ~ Concatenate with the rows;
  .+M      ~ For each list;
  .+       ~ Get the deltas;
 s         ~ Flatten the list of deltas;
s          ~ Get the sum;
           ~ Print Implicitly;

Jelly, 5 bytes

;ZIẎS

Try it online!

There are a couple ASCII-only versions too: ;ZIFS ;ZISS

SOGL V0.12, 9 bytes

:⌡-≤H⌡-¹∑

Try it Here! (→ added because this takes input on the stack)

Explanation:

:          duplicate ToS
 ⌡         for each do
  -          get deltas
   ≤       get the duplicate ontop
    H      rotate it anti-clockwise
     ⌡     for each do
      -      get deltas
       ¹   wrap all of that in an array
        ∑  sum

05AB1E, 6 bytes

ø«€¥OO

Uses the 05AB1E encoding. Try it online!