C#, 95 bytes

using System;class M{public static void Main(string[]a){Console.Error.Write("Hello World!");}}

I'm trying to learn this language but man, it's verbose.

Ungolfed:

using System;

class MainClass {
    public static void Main (string[] args) {
        Console.Error.Write ("Hello World!");
    }
}

Matlab, 21 bytes

error('Hello World!')

Lua5.2, 41 bytes

io.stderr.write(io.stderr,"Hello World!")

Cheeky golfed version of the transpilation of my Moonscript answer.

Moonscript, 40 bytes

io.stderr.write io.stderr,"Hello World!"

CoffeeScript for Lua... because Lua is ugly as hell!

Perl 6, 19 21 bytes

note("Hello World!");

JavaScript, 30 bytes

console.error("Hello, World!")

An alternative JavaScript answer, using console.error.

Forth, 65 bytes

Shorter than factor!

outfile-id stderr to outfile-id ." Hello World!" cr to outfile-id

zsh, 21 20 19 bytes

<<<Hello\ World!>&2

This is a zsh-specific feature; won't work in bash.

Thanks to @FlagAsSpam and @Dennis for a byte each!

Factor, 67 bytes

"Let's use a modernisation of Forth", I said. "It will be fun!" I said.

Yea, the whitespace is needed.

error-stream get [ "Hello World!" print flush ] with-output-stream*

Perl, 25 24 bytes

say stderr"Hello World!"

Requires version 5.10 or later.

Thanks to @FlagAsSpam for a byte!

Rust, 73 72 bytes

use std::io::Write;fn main(){std::io::stderr().write(b"Hello, World!");}

Rust is as long as ever...

Thanks to @ICanHazHats for saving a byte!

Pike, 35 bytes

int main(){werror("Hello World!");}

Looks like C, isn't C. Is interpreted.

We can make this compile as ANSI C like this:

#include <stdio.h>
# ifndef __PIKE__
#   define werror(x) fputs(x, stderr)
# endif

int main() {
  werror("Hello World!");
}

Pascal (FP/GP), 51 bytes

program h;begin writeln(StdErr,'Hello World!');end.

C, 35 bytes

main(){write(2,"Hello World!",12);}

write writes to a file descriptor. STDERR is file descriptor 2, and 12 is the length of the string "Hello World!".

Microsoft Windows Batch (also works in Wine), 22 bytes

echo Hello World! 1>&2

Mathematica, 35 bytes

"stderr"~WriteString~"Hello World!"

Quite simple.

Python 3, 33 bytes

open(2,'w').write('Hello World!')

Chapel, 29 bytes

stderr.write("Hello World!");

This language is probably overkill.

PHP, 36 bytes

I don't know much PHP, but here goes...

<?php fwrite(STDERR,"Hello World!");

Tested on ideone.com.

JavaScript, 19 bytes

throw"Hello World!"

Yes. It's that simple. Try it in the browser console on any page.

Go, 39 38 37 70 69 bytes

package m;import."os";func main(){Stderr.WriteString("Hello World!")}

Bash, 24 21 20 bytes

Saved 3 bytes thanks to @Neil, then another from @Dennis!

echo Hello World!>&2 

>&2 pushes the output to STDERR.

Japt, 17 bytes

Currently winning! Let's hope Jelly doesn't pop in with an 11-byte answer...

Ox`È*w'HÁM W?ld!'

The ? should be the literal byte 8E. Test it online!

(STDERR is found below the "Upload a file" line in red text.)

How it works

  `È*w'HÁM W?ld!'  // Decompress this string. Returns "throw'Hello World!'"
Ox                 // Evaluate as JavaScript code. Throws the error.
                   // Implicit: Error is caught by interpreter and sent to STDERR.

Alternate version:

$throw$`HÁM W?ld!

Python 2, 36 bytes

import os
os.write(2,'Hello World!')

Despite being based on a Unix syscall, this works as well on Windows.

C++, 53 bytes

#include<iostream>
main(){std::cerr<<"Hello World!";}

You can try it online if you feel so inclined.

Ruby, 18 bytes

warn"Hello World!"

This only works if warnings are on (which is the default).

Java 8, 75 37 bytes

Thanks to @GamrCorps for slicing the code size almost exactly in half and showing me how to use lambda!

()->System.err.print("Hello World!");

Using interface because it makes the main declaration shorter. c:

AppleScript, 19 bytes

error"Hello World!"

...pretty self-explanatory.

R, 23 bytes

message("Hello World!")

The message function writes to STDERR.

Julia, 28 bytes

print(STDERR,"Hello World!")

Does what it looks like it does. STDERR is a built-in constant that refers to the standard error stream and print takes an optional argument specifying the stream.

Note that using error() prints out a bunch of extra garbage.

AutoIt, 33 bytes

ConsoleWriteError("Hello World!")