Befalse, 43 bytes

" !!ddllrrooWW  ,,oolllleeHH"!: ::::: .!.#;

Try it online!

Every odd byte is exactly this Hello World program, so this solution is the theoretical optimum until the original Hello World is golfed further.

• ::::: <code>; runs <code> exactly 13 times. Note a space after the group of colons.
• In order to print Hello, World! from the duplicated string, the output command . and the discard command # must alternate. The original HW contains two .s in the <code> section, so I decided to skip the second using ! (skip) and insert # at the only left extra space.
• The extra leading : can be skipped with !.

Perl 5 + -p0513, 43 bytes

$ _ ="''H'e'l'l'o',' 'W'o'r'l'd'!''";s#'##g

Try it online!

Every odd byte, 22 bytes

$_='Hello, World!';##g

Try it online!

JavaScript (V8), 66 bytes

"";alert`Hello, World`; " ; a l e r t ` H e l l o ,   W o r l d `"

Try it online!

String as comment, nothing wrong

x86 machine code (DOS), 53 bytes

Full:

00000000: ba 00 0d b0 01 b0 b4 b0 09 b0 cd b0 21 b0 c3 be  ............!...
00000010: 1a 01 ad cd 29 3c 21 75 f9 c3 48 00 65 00 6c 00  ....)<!u..H.e.l.
00000020: 6c 00 6f 00 2c 00 20 00 57 00 6f 00 72 00 6c 00  l.o.,. .W.o.r.l.
00000030: 64 00 21 00 24                                   d.!.$

Even bytes only, 27 bytes

00000000: ba 0d 01 b4 09 cd 21 c3 1a ad 29 21 f9 48 65 6c  ......!...)!.Hel
00000010: 6c 6f 2c 20 57 6f 72 6c 64 21 24                 lo, World!$

Assembly (NASM syntax)

    [org 0x100]
start:
    ; Encode the half version using the defacto hello world with some
    ; harmless mov instructions. I haven't found a good use for these bytes
    ; yet (point of interest: get 13 in CX or (hello - start) in AX.
    ; mov dx, hello
    ; The offset is halved, then shifted left 8 to get it to encode in the
    ; upper byte.
    ;
    ; As a 3 byte opcode, this jumps the full variant to an odd address,
    ; causing a desync.
    mov dx, ((hello - start) / 2) << 8
    mov al, 0x01
    ; mov ah, 0x09
    mov al, 0xB4
    mov al, 0x09
    ; int 0x21
    mov al, 0xCD
    mov al, 0x21
    ; ret
    mov al, 0xC3

    ; Now for the thicc code. Instead of INT 21:09H, which expects a byte
    ; string terminated with '$', I print each char manually using INT 29H
    ; since I can do that with LODSW.

    ; Get the address of str into SI
    mov si, hello
.loop:
    ; Load one padded UTF-16 character into AX, and advance SI.
    lodsw
    ; Print AL using INT 29H.
    int 0x29
    ; Check for the last character
    cmp al, '!'
    jne .loop
    ; Return to exit
    ret
    ; On the minified version, this is a $-terminated string for INT 21:09H.
    ; db "Hello, World!$"
    ; On the non-minified version it is just dummy thicc.
hello:
    dw __UTF16__("Hello, World!") ; Unicode strings are proof that nasm > gas
    ; this one doesn't need padding
    db '$'

The full variant does this:

    [org 0x100]
start:
    ; bunch of pointless mov instructions

    mov si, str
.loop:
    lodsw
    int 0x29
    cmp al, '!'
    jne .loop
    ret
str:
    dw __UTF16__("Hello, World!")

while the small variant does this

    [org 0x100]
start:
    mov dx, str
    mov ah, 0x09
    int 0x21
    ret
    ; Unexecuted nonsense here
str:
    db "Hello, World!$"

This does not attempt to do even bytes only, unlike Hagen von Eitzen's solution. This allows me to simplify it greatly.

As the comments say, I use two methods for printing the string. For the larger code, I loop over the padded string with LODSB; INT 29H, and for the shorter code I use INT 21:09H to just print the string directly from memory. This means I don't need to use any complex tricks.

I feel like there is some improvement possible, as I think I could take better advantage of the nops and padding.

Python 3, 60 bytes

#
exec(#
'p_r_i_n_t_(_"_H_e_l_l_o_,_ _W_o_r_l_d_!_"_)'[::2])

Try it online!

The [::2] slice the string to have 'print("Hello, World!")'. Then we execute it with exec

Every other char :

#ee(
print("Hello, World!")[:]

This prints Hello, World! and then error due to the slice on a None object

Zsh, 35 bytes

e\c\h\o  \H\e\l\l\o\,\ \W\o\r\l\d\!

Attempt This Online!

AtmtTi nie

JavaScript, 138 bytes

A lot of people used comments and I found it too easy so here's one entry without comments:

c_n_o_e_l_g ="c o n s o l e . l o g"; console.log ( "_H_e_l_l_o_,_ _W_o_r_l_d_!".replaceAll( "_","") ) ;[ ' s l i c e ' ]+( 0 , - 9 )+ ")"

Taking every other character out:

cnoelg=console.log;cnoelg("Hello, World!.elcAl _,"  ['slice'](0,-9) )

Important

This code uses String.prototype.replaceAll(), which is a fairly recent feature, and thus wont work in older browsers/NodeJS versions (NodeJS v15.0.0 and above is required, for example).

Explanation

1. First it create an alias of console.log named c_n_o_e_l_g which returns the same as console.log when every other character is removed.
2. Then console.log is called (being replaced with cnoelg when converted).
3. Inside console.log there's "_H_e_l_l_o_,_ _W_o_r_l_d_!" in which every underscore (_) is removed, but not in the converted version (they will be removed anyway because in even position); because of the clever placement of the last double quote ("), the string will look like "Hello, World!.elcAl _,".
4. Since "Hello, World!.elcAl _," needs to be cleaned up, a slice() is applied with the brackets notations ([]), removing the last 9 characters (".elcAl _,").
5. But we don't want any modification to happen to the not converted version, so a ) closes the console.log being removed when converted alongside a ; (since in even position). Then it will apply correctly to the converted version but nothing will happen to the not converted one, since it's a value not assigned to anything.
6. This consist of [ ' s l i c e ' ], ( 0 , - 9 ) and ")", concatenated with a + sign (removed during conversion), so that the script stays valid. The last ")" is there to close the console.log in the converted version.

Backhand, 33 bytes

>'l"o'"'l','!'e' 'd'H'WHl'"'o.r'H

Try it online!

Only works in the latest interpreter, so the entire interpreter is copied into the TIO link (TIO's Backhand seems to not have ' implemented).

This one has only one copy of "Hello, World!" in the entire code.

>'l"o'"'l','!'e' 'd'H'WHl'"'o.r'H   Full program
>  "  "                             Push empty string (no-op)
         '  !  '  d  '  l  '  r     Push "!dlr" (`'` pushes next char)
    o  '  ,  '     '  W  '  o  '    Bounce off and push "oW ,o"
 'l  '  l  '  e  '  H               Bounce off and push "lleH"
                       H            Halt and print entire stack

>lo"l,!e dHWl"orH   Halved program
>  "                Start string mode
      !  d  l  r    Push "!dlr"
  o  ,     W  o     Bounce off and push "oW ,o"
 l  l  e  H  "      Bounce off and push "lleH" and end string mode
                H   Halt and print entire stack

Backhand, 34 bytes

>>lvo""!ld,l!reo Wd H,Woll"loerHH"

Try it online!

Halved:

>lo"l,!e dHWl"orH

Try it online!

This one works in the TIO version, and has two disjoint program paths. The full program has its step size reduced at an odd position, which makes the rest of its path totally invisible to the halved program. (Bonus points for looking totally jumbled, I guess.)

>>lvo""!ld,l!reo Wd H,Woll"loerHH"   Full program
>  v                                 Reduce step size to 2
     " ! d l r o W   , o l l e H "   Push the string
                               H     Halt and print

>lo"l,!e dHWl"orH   Halved program
>  "  !  d  l  r    →
  o  ,     W  o     ←
 l  l  e  H  "      →  Push the string
                H   Halt and print

Vyxal, 29 bytes

`HHeelllloo,,  WWoorrlldd!!`√

Try it Online!

`Hello, World!√

Try it Online!

Without using the kH builtin, which returns "Hello, World!"

`HHeelllloo,,  WWoorrlldd!!`  # String
                            √ # Get every second character

`Hello, World!  # String (unterminated)
              √ # Ignored due to parsing bug

Ly, 39 [40] bytes

"x!xdxlxrxoxWx x,xoxlxlxexHx"'_p_p[xop]

Try it online!

Thanks to Aaroneous Miller for the -1 improvement hint in the comments.

The alternate code, with all the odd number characters deleted is,

"!dlroW ,olleH"__[o]

Which is straightforward. The only trick used is relying on Ly's willingness to ignore any character is doesn't recognize. So the _ characters (and the x characters in the longer code) can be replaced by any character that's undefined in the language.

The longer code works because the loop running through the stack and printing things, meaning

 [xop]

deleted one character for every one it prints.

Vyxal, 7 bytes

ka`a`kH

Try it Online!

Pretty straightforward: The kH at the end is a builtin for Hello, World!, and that's the only thing that gets implicitly printed.

With characters removed:

k``H

Try it Online!

This makes use of a quirk in Vyxal where you can put a string in the middle of a constant diagraph, and the constant will be pushed after the string, making this equivalent to ``kH, which pushes the same builtin as before and prints it.

Pari/GP, 47 bytes

p r i n t ( H e l l o "\,\ """W o r l d "\!""")

Try it online!

print(Hello", "World"!")

Try it online!

A boring answer. In PARI/GP, all spaces not contained in a string are simply ignored. Hello and World are variables that can be used in polynomials.

AWK, 87 bytes

BEGIN {print"Hello, World!" }
B#E#G#I#N# #{#p#r#i#n#t#"#H#e#l#l#o#,# #W#o#r#l#d#!#"# #}

Try it online!

The resulting code is:

BGN{rn"el,Wrd"}BEGIN {print"Hello, World!" }

Because the BGN variable is false (not assigned), the first part of the code has no effect.

dc, 49 bytes

[Hello, World!]p #[[HHeelllloo,,  WWoorrlldd!!]]p

Try it online!

C (MSVC), 60 bytes

As Stack Exchange doesn't like me posting null bytes, here is an xxd -g1:

00000000: 6d 00 61 00 69 00 6e 00 28 00 29 00 7b 00 70 00  m.a.i.n.(.).{.p.
00000010: 75 00 74 00 73 00 28 00 22 00 48 00 65 00 6c 00  u.t.s.(.".H.e.l.
00000020: 6c 00 6f 00 2c 00 20 00 57 00 6f 00 72 00 6c 00  l.o.,. .W.o.r.l.
00000030: 64 00 21 00 22 00 29 00 3b 00 7d 00              d.!.".).;.}.

For an infinitely less readable textual version, with 0 representing a raw 0x00 byte:

m0a0i0n0(0)0{0p0u0t0s0(0"0H0e0l0l0o0,0 0W0o0r0l0d0!0"0)0;0}0

Explanation

I'll see your small brain comment abuse and raise you one cosmic brain text encoding abuse. 😏

Unlike GCC and Clang, MSVC supports files encoded in UTF-16.

That is what we abuse. MSVC will """incorrectly""" auto-detect this file as-is as UTF-16LE, causing it to """incorrectly""" read it as this, encoded in UTF-16LE:

main(){puts("Hello, World!");}

If we remove every other character (a.k.a. removing every 00 byte), it treats it as UTF-8 (well, technically, the current SBCS/MBCS codepage), and it is compiled as you would probably expect, as the codepoints are now the same as they were in UTF-16LE:

main(){puts("Hello, World!");}

You may say "but this is UTF-16, it should be mi({us"el,Wrd";", but I say that U+0000 is a valid UTF-8 codepoint, and it just looks like UTF-16 by """coincidence""".

05AB1E, 29 bytes

”™ ,ï‚‚ï ! ”# ¦2 ä ø¨øJð ý

Try it online!

Explanation

”™ ,ï‚‚ï ! ”                # push the string "Weekly Hello , Changed World ! "
               #               # split on spaces
                               # RESULT: ['Weekly','Hello',',','Changed','World','!','']
                ¦              # remove the first element (Weekly)
                 2ä            # split in 2 parts
                               # RESULT: [['Hello', ',', 'Changed'], ['World', '!', '']]
                   ø           # zip
                               # RESULT: [['Hello', 'World'], [',', '!'], ['Changed', '']]
                    ¨          # remove the last element
                     ø         # zip
                               # RESULT: [['Hello', ','], ['World', '!']]
                      J        # join each inner list
                       ðý      # join on space

After removing every other character we are left with the code

”Ÿ™,‚ï!” 2äøøðý

Try it online!

Explanation

”Ÿ™,‚ï!”       # push the string "Hello, World!"
        2ä     # split in 2 parts
               # RESULT: ['Hello, ', 'World!']
          ø    # zip, as the string has an odd length the space is lost
               # RESULT: ['HW', 'eo', 'lr', 'll', 'od', ',!']
           ø   # zip again
               # RESULT: ['Hello,', 'World!']
            ðý # join on space

05AB1E, 25 bytes

”Ÿ™,‚ï!”,””q”
Ÿ
™
,
‚
ï
!

Try it online!

”Ÿ™,‚ï!”,””q”Ÿ™,‚ï!  # full program (newlines aren't important here)
        ,            # output...
”Ÿ™,‚ï!”             # "Hello, World!"
         ””          # push empty string
           q         # exit program
            ”Ÿ™,‚ï!  # push "\nŸ\n™\n,\n‚\nï\n!" (never executed)

Every other character, 13 bytes

”™‚!,””Ÿ™,‚ï!

Try it online!

”™‚!,””Ÿ™,‚ï!  # full program
”™‚!,”         # push "Situation!,"
      ”Ÿ™,‚ï!  # push "Hello, World!"
               # implicit output

Julia, 59 bytes

".H.e.l.l.o. .W.o.r.l.d.!\" | > p r i n t #"[2:2:24]|>print

Shortened version :

"Hello World!"|>print#[::4|pit

Try it online!

Javascript, 67 bytes

/**/alert`Hello, World`// * / a l e r t ` H e l l o ,   W o r l d `

Every second letter removed:

/*aetHlo ol`/*/alert`Hello, World`

Just like many other answers, this exploits comments.

Lost, 113 bytes

v<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>.%?"Hello, worldvU"-+@
v>v%v?v"vHvevlvlvov,v 
wvovrvlvdvvvUv"v-v+v@

Try it online!

With removal, 57 bytes

v<<<<<<<<<<<<<<<<<<<<<
.?Hlo olv"+
>%?"Hello, worldvU"-+@

Try it online!

Java (JDK), 123 bytes

/*/ */System.out.print("Hello, world!"); ///\ u 0 0 0 d S y s t e m . o u t . p r i n t ( " H e l l o_,_ _w o r l d ! " ) ;

Try it online!

This isn't a full program - you'll have to paste it into a method somewhere.

Every other character, 62 bytes

//*Sse.u.rn(Hlo ol!) /\u000dSystem.out.print("Hello, world!");

Try it online!

It takes advantage of the way the compiler turns unicode escapes into their corresponding characters.

Keg, 47 39 35 bytes

`1H1%1;1,1 1c1¡1;1!1`_`H%;, c¡;!` $

Try it online!

And, unless I've somehow misread the challenge, the program with alternating characters:

-hd, 18 bytes

`H%;, c¡;!``%,c;`$

Try it online!

MAWP, 169 bytes

9/8/W/;/5/5/W/4/W/1/M/;/9/3/W/4/W/;/9/3/W/4/W/;/9/4/W/1/M/3/W/;/5/8/W/4/M/;/8/4/W/;/9/8/M/5/W/2/M/;/9/4/W/1/M/3/W/;/9/9/M/1/M/6/W/;/9/3/W/4/W/;/5/5/W/4/W/;/9/2/M/3/W/;/.

Try it!

Code with every other character removed:

98W;55W4W1M;93W4W;93W4W;94W1M3W;58W4M;84W;98M5W2M;94W1M3W;99M1M6W;93W4W;55W4W;92M3W;.

Try It!

Befunge-93, 43 41 bytes

Coming back to this a few years later, I found a way to avoid running those unsupported instructions, meaning this can even be run on implementations that treat those differently.

"!!ddllrrooWW  ,,oolllleeHH"""> :$# ,#__@

Try it online!

Second version

"!dlroW ,olleH">:#,_@

is the shortest possible "Hello World!". This therefore is the shortest possible answer to this question for Befunge-93.

Try It Online

Rust, 85 bytes

| | /**/print!("Hello, World!")//___*//_p_r_i_n_t_!_(_"_H_e_l_l_o_,_ _W_o_r_l_d_!_"_)

Try it online

Every other character:

||/*pit(Hlo ol!)/_*/print!("Hello, World!")

Try it online

Full program, 113 bytes

/**/fn main(){print!("Hello, World!")}//__*//_f_n_ _m_a_i_n_(_)_{_p_r_i_n_t_!_(_"_H_e_l_l_o_,_ _W_o_r_l_d_!_"_)_}

Try it online

Every other character:

/*f an)pit(Hlo ol!)/_*/fn main(){print!("Hello, World!")}

Try it online

Stax, 33 bytes

"zHzezlzlzoz,z zWzozrzlzdz!z"'z -

Run and debug it

Every other character, 17 bytes

"Hello, World!"z-

Run and debug it

In the first version, 'z - remove all zs from the string. In the second version, z by itself is empty set, and z- is noop.

Java 8, 245 243 241 239 238 237 bytes (full program)

/**/interface M{static void main(String[]a){System.out.print("Hello, World!");}}//**/ iinntteerrffaaccee  MM{{ssttaattiicc  vvooiidd  mmaaiinn((SSttrriinngg[[]]aa)){{SSyysstteemm..oouutt..pprriinntt((""HHeelllloo,,  WWoorrlldd!!""));;}}}

Try it online.

After removing every other character:

/*itraeMsai odmi(tig])Sse.u.rn(Hlo ol!)}/*/interface M{static void main(String[]a){System.out.print("Hello, World!");}}

Try it online.

-2 bytes (243 → 241) thanks to @OlivierGrégoire.
-2 bytes (241 → 239) by creating a port of @SuperStormer's JavaScript answer.

Java 8, 63 bytes (lambda function)

/**/v->"Hello, World!"//**/vv-->>""HHeelllloo,,  WWoorrlldd!!""

Try it online.

After removing every other character:

/*v>Hlo ol!/*/v->"Hello, World!"

Try it online.

Explanation:

Utilizes //abc single-line comments and /*abc*/ multi-line comments.
See the Java-highlighting of the two programs/functions to see how these type of comments are used.

Bash, 36 bytes

e\c\h\o  "H"e"l"l"o"," "W"o"r"l"d"!"

With every other character removed:

echo Hello, World!

Invalid escape sequences will be ignored, so \c\h\o makes no difference from cho.

Cubically, 155 bytes

R U + 4 3 2 @ 6 + 5 0 - 4 @ 6 + 3 - 4 @ 6 @ 6 + 1 - 0 0 @ 6 - 3 3 1 @ 6 - 0 0 @ 6 + 4 1 1 0 @ 6 + 0 0 0 0 @ 6 + 1 - 0 0 @ 6 - 0 @ 6 - 2 + 4 @ 6 - 3 3 1 @ 6

The new parser strips out all spaces ¯\_(ツ)_/¯ so either way the code is interpreted like this:

RU+432@6+50-4@6+3-4@6@6+1-00@6-331@6-00@6+4110@6+0000@6+1-00@6-0@6-2+4@6-331@6

Try it online! (spaced)

Try it online! (every other character)

BotEngine, 91 bytes

 vv 
 v < 
 eHH 
 eee 
 ell 
 ell 
 eoo 
 e,, 
 e   
 eWW 
 eoo 
 err 
 ell 
 edd 
 e!! 
 P

Befunge-93, 51 bytes

Inspired by Martin's Cardinal solution, I thought it might be fun to try and do something vertical in Befunge as well. It's a little more complicated in Befunge though.

?
"
!
d
l
r
o
W

,
o
l
l
e
H
" :
>>:v$
#^,__@@
v >

Try it online!

We start with a ?(random direction) command, which can potentially send the instruction pointer in any direction. Left and right will just wrap around back to the start, and going up will hit the v at the bottom of the program and reflect back to the start as well. So while we may bounce around for a while, we'll eventually be forced down at some point, pushing the "Hello, World!" string (in reverse) onto the stack. Which brings us to this sequence:

>>:v
 ^,__@

This is just a simple output loop that writes out the string on the stack. The only thing out of the ordinary is the extra _ (branch) when the loop terminates. But because there are only zeros on the stack at that point, the second test will safely continue on to to the right, finishing with the @ (exit) command.

Once we remove every second character, the code looks like this:

?"!dlroW ,olleH":>:$#,_@v>

Try it online!

Again we start with a ?(random direction) command, and again most directions will just end up wrapping back to the start. So eventually we're forced to go to the right, pushing "Hello, World!" (in reverse) onto the stack. Which then brings us to this sequence:

:>:$#,_@

Again this is just a loop outputting the string on the stack. It's very similar to the classic inline print loop, but we've got an extra : (dup) command, and thus require a $ (drop) to cancel it out.

Cubically v2.1, 222 bytes

+0503 @@6 :22 //1 +050501 @@6 :55 +0502 @@6@6 :33 //1 +050502 @@6 :55 +03 //1 +04 @@6 :55 //1 +03 @@6 :55 +01 //1 +0504 @@6 :33 //1 +050502 @@6 :55 +01 //1 +050502 @@6 :55 +0502 @@6 :11 //1 +050501 @@6 :55 +01 //1 +03 @@6

Try it online!

Every other letter:

+53@6:2/1+551@6:5+52@66:3/1+552@6:5+3/1+4@6:5/1+3@6:5+1/1+54@6:3/1+552@6:5+1/1+552@6:5+52@6:1/1+551@6:5+1/1+3@6

Try it online!

Ly, 38 bytes

"""!!ddllrrooWW  ,,oolllleeHH" [po ]

Try it online!

Explanation:

"""HHeelllloo,,  WWoorrlldd!!" [po ]

"""           # open a string literal, close it, then open one again
   ...        # push "HHeelloo,,  WWoorrlldd!!" backwards
      "       # close string literal
         [    # until the stack is empty
          po  # pop off the stack, then output
            ] # end loop

Every other character:

"!dlroW ,olleH"[o]

Try it online!

Explanation:

"!dlroW ,olleH"[o]

"!dlroW ,olleH"    # push "Hello, World!" backwards
               [o] # output until the stack is empty

Powershell, 46 bytes

Full version:

#
"Hello, World"
#" H e l l o ,   W o r l d " 

Try it online here

Every other character

#"el,Wrd
"Hello, World"

Try it online here

Brainfuck 156 bytes

-
-
<
-
<
<
+
[
+
[
<
+
>
-
-
-
>
-
>
-
>
-
<
<
<
]
>
]
<
<
-
-
.
<
+
+
+
+
+
+
.
<
<
-
.
.
<
<
.
<
+
.
>
>
.
>
>
.
<
<
<
.
+
+
+
.
>
>
.
>
>
-
.
<
<
<
+
.

x86 machine code, 162 bytes

PROG.COM Download and run it in MS-DOS emulator, DOSBox for example.

90 B3 B4 B4 02 90 90 B3 B2 B2 48 90 90 B3 CD CD 21 90 90 B3 B2 B2 65 90 
90 B3 CD CD 21 90 90 B3 B2 B2 6C 90 90 B3 CD CD 21 90 90 B3 CD CD 21 90 
90 B3 B2 B2 6F 90 90 B3 CD CD 21 90 90 B3 B2 B2 2C 90 90 B3 CD CD 21 90 
90 B3 B2 B2 20 90 90 B3 CD CD 21 90 90 B3 B2 B2 77 90 90 B3 CD CD 21 90 
90 B3 B2 B2 6F 90 90 B3 CD CD 21 90 90 B3 B2 B2 72 90 90 B3 CD CD 21 90 
90 B3 B2 B2 6C 90 90 B3 CD CD 21 90 90 B3 B2 B2 64 90 90 B3 CD CD 21 90 
90 B3 B2 B2 21 90 90 B3 CD CD 21 90 90 B3 CD CD 20 90

after removal MINI.COM Download

90 B4 02 90 B2 48 90 CD 21 90 B2 65 90 CD 21 90 B2 6C 90 CD 21 90 CD 21 
90 B2 6F 90 CD 21 90 B2 2C 90 CD 21 90 B2 20 90 CD 21 90 B2 77 90 CD 21 
90 B2 6F 90 CD 21 90 B2 72 90 CD 21 90 B2 6C 90 CD 21 90 B2 64 90 CD 21 
90 B2 21 90 CD 21 90 CD 20

How to run?

Install DOSBox, for Ubuntu/Debian

sudo apt install dosbox

Run it

dosbox

In DOSBOX

mount c /home/user/path/to/your/directory
c:
PROG.COM
MINI.COM

How does it works?

Machine operation codes represents assembly language instructions.

In MS-DOS to print char you will set registers and make interrupt. AH register will be 0x02, DL register contains your char. Interrupt vector is 0x21.

mov ah,0x2  ;AH register to 0x2 (B4 02)
mov dl,0x48 ;DL register to "H" (B2 48)
int 0x21    ;0x21 interrupt     (CD 21)

MS-DOS COM file tiny model is good choise, because it doesn't have any headers. It is limited by 64K, but in our case it doesn't matters.

To stop program use 0x20 interrupt

int 0x20    ;0x20 interrupt     (CD 20)

Magic

If you want to execute 0xAB opcode command with one parameter 0xCD, you write

AB CD

In PROG.COM

90 B3 AB AB CD 90
nop         ; No operation (90)
mov bl,0xb4 ; BL register to AB (B3 AB)
AB CD command (AB CD)
nop         ; No operation (90)

In MINI.COM

90 AB CD
nop         ; No operation (90)
AB CD command (AB CD)

It is equal machine codes, if you don't use BL register.

Generator

Convert text file with hex to hex binary

cat hex_file | xxd -r -p > exec.com

function byte2hex(byte){
	var ret=byte.toString(16).toUpperCase();
	return ret.length==1 ? "0"+ret : ret;
}

function str2hex(str){
	var ret = [];
	for(var i=0;i<str.length;i++){
		ret.push(byte2hex(str.charCodeAt(i)));
	}
	return ret;
}

function genCode(hexArr){
	var ret = [["B4","02"]];
	for(var i=0;i<hexArr.length;i++){
		if(hexArr[i]!=hexArr[i-1]){
			ret.push(["B2",hexArr[i]]);
		}
		ret.push(["CD","21"]);
	}
	ret.push(["CD","20"]);

	return ret;
}


function magicCode(str){
	var ret=[""];
	var code=genCode(str2hex(str));

	for(var i=0;i<code.length;i++){
		ret.push("90 B3 "+code[i][0]+" "+code[i][0]+" "+code[i][1]+" 90");
		if(i%4==3){ret.push("\n");}
	}
	return ret.join(" ");
}

function magicCodeMinified(str){
	var ret=[""];
	var code=genCode(str2hex(str));

	for(var i=0;i<code.length;i++){
		ret.push("90 "+code[i][0]+" "+code[i][1]);
		if(i%8==7){ret.push("\n");}
	}
	return ret.join(" ");
}

var str=prompt("string","Hello, world!");
var out="PROG.COM\n" + magicCode(str)+"\n\nMINI.COM\n"+magicCodeMinified(str);

document.write(out.replace("\n","<br>"));
alert(out);

dc, 52 bytes

0#[#H#e#l#l#o#,# #W#o#r#l#d#!#]#P#q
[Hello, World!]P

Try it online!

The first line puts a zero on the stack, and then the rest of the line is commented out. The second line prints 'Hello, World!'

With every even character removed, all the #s in the first line disappear, and the line is no longer commented out. It then prints 'Hello, World!' and quits before hitting the second line.

Ruby, 58 bytes

$/>%< <> > ;puts a= %w( H e l l o ,\  W o r l d ! )* "#{}"

Which becomes:

$><<  pt =%(Hello, World!) #}

I borrowed the puts/assignment trick from @G B's solution

Explanation:

The first version compares the ruby global $/ which evaluates to \n to the string " <> " and does nothing with it. Then it creates an array of all the characters in "Hello, World!" and joins them by multiplying by empty string

The second pushes the return value of assignment of the string "Hello, World!" (which happens to be "Hello, World!" directly to $> which is the global for standard out.

Try it online!

05AB1E, 32 bytes

"
H
e
l
l
o
,

W
o
r
l
d
!
"¶ K

Basically the "s make a string, and the ¶ K removes all newlines.

Try it online!

Version with even characters removed, 16 bytes.

"Hello, World!" 

Try it online!

C, 111 101 bytes

10 bytes saved thanks to Jasen

Full version:

//*
main(){puts("Hello, World!");}
//**// m a i n ( ) { p u t s ( " H e l l o ,   W o r l d ! " ) ; }

Every other character

/*mi({us"el,Wrd";
/*/main(){puts("Hello, World!");}

x86 machine code, 73 bytes

Inspired by Евгений Новиков's solution, I thought it should be doable with less tricks, i.e., just jumping around to otherwise "disjoint" codes for all three variants. I'm still trying with a smart variant that uses lodsb; lodsb as central point (so only one string constant is needed for all variants)

EB 14 00 00 8A 8A 17 16 01 01 B4 B4 09 09 CD CD
21 21 CD CD 20 20 8A 1f 01 B4 09 CD 21 CD 20 48
65 6c 6c 6f 2c 20 57 6f 72 6c 64 21 00 48 48 65
65 6c 6c 6c 6c 6f 6f 2c 2c 20 20 57 57 6f 6f 72
72 6c 6c 64 64 21 21 00 00

If I recall correctly from my childhood days, COM's tiny model starts with DS=CS=SS and the code is loaded beginning from CS:0100h. I do not assume it is guaranteed that the code is loaded into a zeroed memory block (if it were guaranteed, I could drop two bytes).

Disassembly of the long code should be

  JMP *+14h
  ; 20 irrelevant bytes
  MOV DX,message
  MOV AH,09h
  INT 21h;  print string pointed to by DS:DX
  INT 20h;  exit program
message:
  DB "Hello, World!\0"
  DB "HHeelllloo,,  WWoorrlldd!!\0\0"

Disassembly of odd code

  JMP *+00h
  MOV DX,message
  MOV AH,09h
  INT 21h;  print string pointed to by DS:DX
  INT 20h;  exit program
  ; some irrelevant bytes
message:
  DB "Hello, World!\0"

Disassembly of even code:

  ADC AL,00h
  MOV DX,message
  MOV AH,09h
  INT 21h;  print string pointed to by DS:DX
  INT 20h;  exit program
  ; some irrelevant bytes
message:
  DB "Hello, World!\0"

LOGO, 63 bytes

 ;p r [ H e l l o ,   W o r l d ! ] e r [
 pr[Hello, World!] ;]

With characters at even index removed

 pr[Hello, World!]er[ rHlo ol! ]

Explanation:

; starts an inline comment in Logo. Therefore the first program is equivalent to

 pr[Hello, World!] 

which obviously print "Hello, World!".

In the second program, a function er is invoked. er stands for erase, receive a list in a suitable format as input, and erase some contents specified in the list (the details does not matter). The key point is that er does not return anything, so it is a shorter alternative to ignore, which ignores its contents.

Java 8, 123 122 bytes

Here is the full code

/**/()->System.out.print("Hello, World!")//_*_/_(_)_-_>_S_y_s_t_e_m_._o_u_t_._p_r_i_n_t_(_"_H_e_l_l_o_,_ _W_o_r_l_d_!_"_)_

Try it online here

Here is the 'every other' code

/*(-Sse.u.rn(Hlo ol!)/*/()->System.out.print("Hello, World!")

Try it online here

,,,, 34 bytes

 2"Hteoltlaol,l yWhourmladn!! "⟛

On removing the even numbered characters...

 "Hello, World!"

Explanation

With all the characters:

 2"..."⟛

               no-op
 2             push 2 to the stack
  "..."        push "Hteoltlaol,l yWhourmladn!! " to the stack
       ⟛      pop 2 and the string and push every 2nd character of the string
               implicit output

Without the even numbered characters:

 "..."

               no-op
 "..."         push "Hello, World!" to the stack
               implicit output

JavaScript (ES6), 65 bytes

Latecomer to the game...

01?alert`Hello, World!` : a_l_e_r_t ` H e l l o ,   W o r l d ! `

Becomes:

0?lr`el,Wrd`:alert`Hello, World!`

Obviously, neither the a_l_e_r_t function nor the lr function does exist. That's OK, because their respective code paths are never executed.

Demo

let code0 = "01?alert`Hello, World!` : a_l_e_r_t ` H e l l o ,   W o r l d ! `";
let code1 = code0.replace(/../g, s => s[0]);

console.log(code0);
eval(code0);
console.log(code1);
eval(code1);

dc, 112

 6   C * P
A d * 1 + d P
7 + d P
d P
3 + d P
B   4 * d P
C - P
F   6 * 3 - P
d P
3 + d P
6 - d P
8 - P
B   3 * P

All odd-indexed characters are whitespace that don't affect the output of the program. Character values are built arithmetically from single digits

Try it online.

Python 2, 73 bytes

print"=#0##Hello, World!"[5:] 
#portionate ""IHeeelallof,A TWookralade!?"

Try it online!

Python 2, unfortunately doesn't have that nifty trick the Python 3 answer used :( So I did the obvious comment trickery. "Try it online" code provides the halved version and execs it to make checking easier.

Halved version:

pit=0#el,Wrd"5] 
print "Hello, World!"

My naive solution

C/C++, 111 105 byes 97 bytes

(6 bytes saved using "puts", inspired by Hawkings)

///
f(){puts("Hello, World!");}
#define x\
_f_(_)_{_p_u_t_s_(_"_H_e_l_l_o_,_ _W_o_r_l_d_!_"_)_;_}

After takeaway

//f)pt(Hlo ol!)}#eiex 
f(){puts("Hello, World!");}

Stacked, 60 bytes

'Hello, World!'  '''H e l l o ,   W o r l d !''' @oouutt out

Try it online!

I generated the second code with this snippet. Second code is:

'el,Wrd' 'Hello, World!' out u

The first one generates two strings, Hello, World! and 'H e l l o , W o r l d !'. The latter is saved to variable oouutt, and the former is printed.

The second one generates two strings, outputs the second, then errors after encountering u.

Python 2, 63 bytes

#
print'Hello, World!'
#p r i n t ' H e l l o ,   W o r l d ! '

Try it online!

After transformation, this becomes:

#pitHlo ol!
print'Hello, World!'

Swift, 77 bytes

/**/print("Hello, World!")//  * / p r i n t ( " H e l l o ,   W o r l d ! " )

Every other character:

/*pit"el,Wrd"/ */print("Hello, World!")

JavaScript (ES6), 66 bytes

/ /;alert`Hello, World!`
'a l e r t ` H e l l o ,   W o r l d ! `'

//aetHlo ol!
alert`Hello, World!`

Pyth, 31 bytes

p% 2"HHeelllloo,,  WWoorrlldd!!

Try it online!

Becomes

p "Hello, World!

Thanks to @CalculatorFeline for pointing out an error and removing one byte.

Japt, 32 28 bytes

Wrote this up and then noticed Oliver beat me to it with the same method in 05AB1E.

Note: The empty line after the comma contains a space character and the empty line after the "W" contains an unprintable.

`
H
e
¥
o
,
 
W

l
d
!
` rR

Test it

With every second character removed, it becomes (with the unprintable still present between the "W" and the "l"):

`He¥o, Wld!`r

Test it

shell (ksh), 58 bytes

 #e c h o   H e l l o ,   W o r l d ! #
echo Hello World!

(in interactive bash, you'd need to add '\' before the '!')

Jelly, 26 25 bytes

““3ḅaė;œ»ḷ“ 3 ḅ a ė ; œ »

Try it online!

After removing every second character, we're left with the following code.

“3a;»“3ḅaė;œ»

Try it online!

How it works

““3ḅaė;œ»ḷ“ 3 ḅ a ė ; œ »  Main link.

““3ḅaė;œ»                  Index into Jelly's dictionary to yield
                           ["", "Hello, World!"]. 
          “ 3 ḅ a ė ; œ »  Index into Jelly's dictionary to yield.
                          " FullERebitingBEfluffiest adoptable".
         ḷ                 Take the left result.

“3a;»“3ḅaė;œ»  Main link.

“3a;»          Index into Jelly's dicrionary to yield " N-".
               Set the argument and the return value to the result.
     “3ḅaė;œ»  Index into Jelly's dicrionary to yield "Hello, World!".
               Set the return value to the result.

APL (Dyalog), 35 34 bytes

-1 thanks to Martin Ender.

'0H0e0l0l0o0,0 0W0o0r0l0d0!0'~ ⍕ 0

Try it online!

'0H0e0l0l0o0,0 0W0o0r0l0d0!0' the message with zeros as removable filler characters

~ except

⍕ formatted (stringified)

0 number zero

Leaving just the odd characters, this becomes 'Hello, World!' .

Lua, 85 bytes

I decided to try making a function copying Lua answer. It's slightly shorter than the comment abuse submission.

paints= p_r_i_n_t----
print("Hello, World!")

paints( " H e l l o ,   W o r l d ! " )

Try it online!

Alternated:

pit=print--pit"el,Wrd"
pit("Hello, World!")

Try it online!

Alternator script is in my other answer.

Japt, 25 23 bytes

Saved 2 bytes thanks to @Shaggy

`\H\e\¥\o\,\ \W\Ž\l\d\!

Try it online!

This seems to be the optimal compressed string that would output Hello, World! while ignoring every other \.

Noodel, 25 bytes

Hðeðlðlðoð,ð¤ðWðoðrðlðdð!

Try it:)

How it works

Hðeðlðlðoð,ð¤ðWðoðrðlðdð! # The ð character breaks the string into an array and pushes the array ["H", "e", "l", "l", "o", ",", "¤", "W", "o", "r", "l", "d", "!"].
                          # Implicitly printed to the screen (¤ is the space).

Without every other character gives you :

Hello,¤World!

<div id="noodel" code="Hðeðlðlðoð,ð¤ðWðoðrðlðdð" input="" cols="12" rows="2"></div>

<script src="https://tkellehe.github.io/noodel/noodel-latest.js"></script>
<script src="https://tkellehe.github.io/noodel/ppcg.min.js"></script>

Mathematica, 62 bytes

P0r0i0n0t0@0"0H0e0l0l0o0,0 0W0o0r0l0d0!0"Print@"Hello, World!"

It returns "0H0e0l0l0o0,0 0W0o0r0l0d0!0" Null P0r0i0n0t0[0], and prints Hello, World! as a side effect. When run as a program (not in the REPL), the return value will not be printed.

After removing every other character:

Print@"Hello, World!"rn@Hlo ol!

It returns Null ol! rn[Hlo], and prints Hello, World!.

Octave, 49 45 bytes

Saved 4 bytes since Octave doesn't require brackets to do indexing.

'HHeelllloo,,  WWoorrlldd!! ' (1:2 : 3 ^ 3)''

Try it online!

And the reduced one:

'Hello, World!'(:    )'

Try it online!

Explanation:

The initial code has the letters in the string duplicated, so that we're left with Hello, World! when every second is removed. Some spaces are added to ensure the brackets and apostrophes are kept.

The indexing is really 1:2:end. There are 27 characters, and we can't use end or 27 since we must remove a character, so we go with 3 ^ 3 instead. When we remove every third character, the indexing becomes (:) (and some additional spaces).

(:) means "flatten and turn into a vertical vector". So, we need to transpose it, using '. We don't need to transpose the string in the original code, but double transposing works, so the first string is transposed twice using '', and the second is transposed just once.

PHP, 63 bytes

#
echo"Hello, World!";
#eecchhoo""HHeelllloo,,  WWoorrlldd!!"";

Try it online!

PHP, 32 bytes

#eh"el,Wrd"
echo"Hello, World!";

Try it online!

Actually, 2 bytes

HH

Explanation:

H, as you might expect, pushes Hello, World! to the stack.

The main program (HH) will encounter the first H and push Hello, World! to the stack. On the second H, however, it will try to use two arguments (as the stack needs to be empty to push Hello, World!) and fail. However, this error will be ignored and then Hello, World! will be implicitly printed.

The second program (H) will push Hello, World! once, and that will be impliclty printed.

This is similar to Fatalize's 2-byte answer, but this doesn't really "cheat".

Try it online!

Self-modifying Brainfuck, 39 bytes

< - [ . <<- ] " e m s p X ! - p m m f I

Try it online!

Explanation

For anyone that is not familir with self-modifying brainfuck: The program itself gets put into registers directly left to where you start out from. This allows for some cool stuff you can't do with regular brainfuck.

This program will move back two registers (one with every second char removed) increment it and output in until the loop stops. The loop will stop once you increment the char that ends the loop itself thus making brainfuck ignore it.

Version without every second character:

 <-[.<-]"emspX!-pmmfI     

Braingolf, 42 40 bytes

#"#H#e#l#l#o#,# #W#o#r#l#d#!#" $_ <$_& @

Try it online!

After removing odd characters:

"Hello, World!"$ $&@

Try it online!

Explanation

Full program:

#"#H#e#l#l#o#,# #w#o#r#l#d#!#" pushes "Hello, World!" (including quotes)

$ adds the silent modifier to the next operator
space does nothing
_ Pops and prints the last item on the stack, but due to silent mode, does not print
< moves the first item on the stack to the end of the stack

This means that $_ <$_ will remove both quotes from the stack

& adds the greedy modifier to the next operator
@ pops and prints the last item on the stack as an ASCII character
  greedy modifier means it prints the entire stack.

With characters removed:

"Hello, World!" Pushes Hello, World!
Spaces are no-ops
&@ Prints entire stack as ASCII
@ is not affected by the silent modifier (apparently)

It feels really cheap using no-ops for this, but it's better than the Java/Python comment solutions imo

Alice, 37 bytes

"_!_d_l_r_o_W_ _,_o_l_l_e_H_"_d_&_o_@

Try it online!

Removing every other character gives:

"!dlroW ,olleH"d&o@

Try it online!

The way this works is that Alice's string mode doesn't push all cell values directly to the stack. Some characters have a special meaning. In particular, _ is a wall which retains it's control flow meaning even when in string mode. But a horizontal wall is a no-op when the IP moves horizontally, so none of those _ do anything. Hence, they can be safely removed.

Ruby, 63 bytes

puts a= 'Hello, World!'#; p u t s ' H e l l o ,   W o r l d ! '

Which becomes:

pt ='el,Wrd';puts'Hello, World!'                    

(Abusing comments like a lot of people do)

Try it online!

TeX, 54 bytes

Full version:

 %H e l l o ,   W o r l d ! \ e n d%
Hello, World!\end

Every other character:

 Hello, World!\end
el,Wrd\n

A bit boring, but for completeness.

Forth, 61 bytes

This uses the fact that \ makes the rest of the line a comment.

 \  . (   H e l l o ,   W o r l d ! )   \
 .( Hello, World! )

Full program

Remove every other byte to get:

  .( Hello, World!) \ (Hlo ol!)

Every other

PHP, 53 bytes

#
echo date(
$e_c_h_o='\H\e\l\l\o\,\ \W\o\r\l\d\!
');

With every other character removed:

#eh ae
echo'Hello, World!';

Pyth, 31 bytes

%y1t"pHEeGlGlhoz,e QwjolrDlBdJ!

eh

Pip, 33 bytes

"0H0e0l0l0o0,0 0W0o0r0l0d0!0"RM 0

Removes the 0s from the string, leaving just Hello, World!. Try it online!

The every-other-character version:

"Hello, World!"M0

Uses the map operator to replace each character in 0 with the string "Hello, World!". Try it online!

><>, 47 bytes

Original:

| v~" H e l l o ,   W o r l d ! "

~o<< ;!!!? l

With every second character removed:

|v"Hello, World!"
o<;!?l

Try them online: original, modified

The original program pushes the characters of "Hello, World!" to the stack (in reverse order) interspersed with spaces, then alternately prints a character and deletes one until the length of the stack is zero. The second program does the same, except the deletion instructions ~ are gone.

If you don't mind halting with an error, we can take a leaf out of Martin Ender's Cardinal book: the modified code is

\"!dlroW ,olleH"!#o#

and the original is the same but with newlines inserted between all the characters, for 39 bytes. Try them online: original, modified.

Javascript, 73 bytes

a ='a0l0e0r0t0`0H0e0l0l0o0,0 0W0o0r0l0d0!0`0/0/';eval(a.replace(/0/g,''))

Constructs a string a with the content a0l0e0r0t0`0H0e0l0l0o0,0 0W0o0r0l0d0!0`0/0/, then removes all 0's to give alert`Hello, World!`//, which is eval'd.

Taking every other character of the program gives

a=alert`Hello, World!`//;vlarpae//,')

which alerts Hello, World using template string syntax the same way as what was eval'd in the full program, then stores the result of the call in a and includes the insightful comment //;vlarpae//,').

Standard ML (MLton), 75 bytes

(*p r i n t " H e l l o ,   w o r l d ! " ( *)print"Hello, world!"(*  * )*)

Try it online!

After removing every other character, we have:

(print"Hello, world!"(*pitHlo ol!( *))

Try it online!

Just exploits the fact that comments are a digraph.

///, 25 bytes

H\e\l\l\o\,\ \W\o\r\l\d\!

Try it online!

With every other character removed:

Hello, World!

Retina, 39 bytes

HHeelllloo,,  WWoorrlldd!!


(.).x?
$1

Try it online!

Taking every other character gives:

Hello, World!
()x
1

Try it online!

The first program creates a string with the greeting duplicated. Then it replaces each pair of characters with the first character. There is also an empty stage that replaces all empty strings with empty strings in between, but that doesn't do anything. The second program fails to match the letter "x" so it doesn't replace anything after creating the greeting.

Perhaps more amusingly, if the third stage is changed slightly the first set of characters doesn't have to be the same message. This could lead to many identical length solutions such as full and halved.

Haskell, 85 bytes

{--}main=putStr"Hello, World!"--} m a i n = p u t S t r " H e l l o ,   W o r l d ! "

Try it online!

Every second character removed:

{-mi=uSrHlo ol!-}main=putStr"Hello, World!"

Try it online!

This exploits the two comment formats in Haskell: {- -} for in-line or multi-line comments and -- to comment the rest of the line.

Haskell, 102 bytes

The full program:

main= putStr"Hello, World!";;
putSt   x ="p u t S t r  \" H e l l o ,   W o r l d !\"";
mmaaiin = main

and with every other character removed:

mi=ptt"el,Wrd";ptt x=putStr "Hello, World!";main=mi

T-SQL, 75 bytes

---
PRINT 'Hello, World!'
/*
-P-R-I-N-T-'-H-e-l-l-o-,- -W-o-r-l-d-!-'
---*/

Single- and multi-line comment abuse, inspired by CalculatorFeline's LUA version.

After removal of all even-numbered characters, some of which are line breaks:

--PIT'el,Wrd'/
PRINT'Hello, World!'--/

Python 3, 61 bytes

rant="partisn't"
print(("HHeelllloo,,  WWoorrlldd!!"""[::2]))

Try it online!

Abusing the fact that print is a function in Python 3 :)

The least partisan solution you'll find here on PPCG.

Becomes

rn=print
rn("Hello, World!"[:])

Jelly, 32 bytes

 2“HHeelllloo,,  WWoorrlldd!! ”m

Try it online!

-2 bytes thanks to @totallyhuman's answer

Now mine is a dupe >.<

Go upvote that one too :D

Help, WarDoq!, 2 bytes

Hi

Try it online!

H prints Hello, World!, i is a no-op.

_{Help, WarDoq! can add two numbers and test for primes, so it is considered as a valid programming language per this meta post.}

C, 125 bytes

xpxuxtxs( ) { }xuxs ( ) { } main( ) {puts ( "Hello, World!" ) ; } mxaxixn ( ) {xpxuxtxs ( " H e l l o ,   W o r l d ! " ) ; }

Try it online!

With even characters removed:

xxxx(){}us(){}mi(){us("el,Wrd");}main(){puts("Hello, World!");}

Try it online!

Brainfuck, 155 bytes

- - < - < < + [ + [ < + > - - - > - > - > - < < < ] > ] < < - - . < + + + + + + . < < - . . < < . < + . > > . > > . < < < . + + + . > > . > > - . < < < + .

Try it online!

Every second character removed:

--<-<<+[+[<+>--->->->-<<<]>]<<--.<++++++.<<-..<<.<+.>>.>>.<<<.+++.>>.>>-.<<<+.

Lua, 89 bytes

--- [ [
print("Hello, World!")
--[[ ] ]
pCrAiLnCtU(L"AHTeOlRlFoE,L IWNoEr:lDd !:"D)
---]]

Try it online! As the syntax highlighting shows, this is massive comment abuse.

Alternate:

--[[pit"el,Wrd"
-[]]print("Hello, World!")--]

Try it online!

And for convenience, a program to convert a program into every other character form: Try it online!

Brachylog, 4 bytes

Ḥ~wḤ

Try it online!

Explanation

~w writes its right variable to STDOUT, and ignores its left argument. Ḥ is "Hello, World!", so this prints Hello, World!.

If we only take the first and third chars, we get Ḥw. In that case w writes its left variable and ignores its right variable, so it also prints Hello, World!.

2 bytes

ḤḤ

Try it online!

This is technically a valid answer, but this unifies the output variable of the program instead of printing to STDOUT, so I guess the 4 bytes program is more in the spirit of the challenge.

Befunge-98, 39 bytes

"" '!'d'l'r'o'W' ','o'l'l'e'H ""c k , @

Try it online!

Every second character removed:

" !dlroW ,olleH"ck,@

Try it online!

Charcoal, 25 bytes

H→e→l→l→o→,→ →W→o→r→l→d→!

Try it online!

If you remove the even characters, you just remove the arrow commands that indicate the direction of the next text, and that leaves the following code:

Hello, World!

Try it online!

That also prints the greeting.

V, 32 bytes

i;H;e;l;l;o;,; ;w;o;r;l;d;!;<esc>;Ó;

Note that <esc> is a single character, e.g. 0x1b

Try it online!

Removing every other character gives:

iHello, world!<esc>Ó

Try it online!

CJam, 32 bytes

"HHeelllloo,,  WWoorrlldd!! "2 %

Try it online!

Taking every other character gives:

"Hello, World!" 

Try it online!

Cardinal, 29 bytes

%
"
H
e
l
l
o
,
 
W
o
r
l
d
!

Try it online!

Removing every other character removes all the linefeeds, which still results in Hello, World!:

%"Hello, World!

Try it online!

The reason this works is that % creates four instruction pointers, moving in each of the four cardinal directions. IPs that leave the source code are simply removed. So in the first case, only the south-going IP remains and in the second case, only the east-going IP remains, all the others are simply dropped. In either case, the executed program is then just "Hello, World!. The " toggles to string mode where each cell is simply printed to STDOUT. We don't need to terminate the string, because leaving the source code still terminates the program.

Note that the same idea works in Beeswax, using * instead of % and ` instead of " (this is because Beeswax was largely inspired by Cardinal but uses a hexagonal grid).

Try it online! (vertical) | | Try it online! (horizontal)