JavaScript (Node.js), 101 bytes - score \$10^{-1000}\$

r=m=>Math.random()*m|0
f=x=>x?r(10)|f(x-1):0
k=0n
while(f(1000))k++
console.log((k%2n?-1n-k:k)/2n+'')

Don't try it online!

Explanation:

Don't try it online because it will time out. So this is a theoretically correct solution. If we replace the constant 1000 in the middle of the program with 5, then it will run fine in TIO.

0 the highest probability which is \$10^{-1000}\$

All other integers have progressively lower probabilities as they extend farther and farther away from 0.

Commented:

r=m=>Math.random()*m|0 //r(m) returns a random integer in the range [0,m-1]
f=x=>x?r(10)|f(x-1):0  //f(x) returns 0 with probability 1/(10^x)  
k=0n                   //initialize k
while(f(1000))k++      //call f(x) until it returns a zero. k keeps count of the number of calls.
console.log((k%2n?-1n-k:k)/2n+'') //convert k from range [0,infinity] to [-infinity,infinity]

Pyth, score of the order \$f_{\varphi(\omega,0)+1}(10^{10^{64}})^{-1}\$

M,-.<hG@H1.<hH@G1+@G1@H1D%ZYI<hZ0R,_hZ@Z1J,1 0V.<1Y=J%gZJY)R,hJ+@J1YVC*GCG=T@%TT1)K0WOT=+K1)IO2K.?_K

Don't try it online!

Pyth, score of the order \$f_{\varphi(\omega,0)+\omega}(4)^{-1}\$

(More precisely around \$f_{\varphi(\omega,0)+4}(10)^{-1}\$)

M,-.<hG@H1.<hH@G1+@G1@H1D%dYI<hd0R,_hd@d1J,1 0V.<1Y=J%gdJY)R,hJ+@J1YVTVTVTVT=T@%TT1;WOT=+Z1)IO2Z.?_Z

Don't try it online!

Original python, but in the Pyth I made changes to increase size:

import random
g=lambda G,H:(G[0]<<H[1]-H[0]<<G[1],G[1]+H[1])
def C(Z,Y):
  if Z[0]<0: return -Z[0],Z[1]
  J=(1,0)
  for N in range(1<<Y):J=C(g(Z,J))
  return J[0],J[1]+Y
T=10
for N in range(T): T=C(T,T)[1]
K=0
while random.randint(0,T):
  K+=1
if random.randint(0,1):
  print(K)
else:
  print(-K)

Explanation

This uses the extremely fast growing functions \$M_n\$ defined here. They're actually my go-to when I need fast-growing functions, since they have a growth rate approaching \$\varphi(\omega,0)\$ and are simple to implement. Since they operate on fractions, I use the representation \$(a,b)=\frac{a}{2^b}\$. I iterate the operation \$n\mapsto M_{2^n}(n)\$ starting from 10 roughly \$10^{10^{64}}\$ times, then output integers in a geometric distribution with parameter \$\frac{1}{n}\$, with signs attached. Thus, the score is \$\frac{1}{n}\$. Yes, you can slightly improve it, but it's so big your improvements are negligible.

Yes, this absolutely dominates every other entry so far.

Charcoal, 26 bytes, score 0.0005

Ｗ‽φ⊞υωＩ⎇‽²Ｌυ±⊕Ｌυ

Try it online! Link is to verbose version of code. 0 and -1 have probabilities of 0.0005, then each subsequent integer has a probability 0.999 times that of the previous. Obviously there are still 75 bytes with which to create some form of arbitrary large expression in order to obtain a lower score, but the resulting code would take longer than the heat death of the universe to run, so it's not quite suitable for TIO. Explanation:

Ｗ‽φ

While a random number 0..999 is non-zero...

⊞υω

...push the empty string to the empty list.

Ｉ⎇‽²Ｌυ±⊕Ｌυ

Randomly print either the length of the list or -1 minus that length.

bash + linux, 1 / big number

cat /dev/random | hexdump -n `b` -ve '/1 "%02u"'

This program will print out 'b' bytes of decimal digits, where b is a program producing a random input number.

(edited to try to be more infinity-ish, although technically still limited by the bash shell)

Python 3, score \$\approx \frac{1}{\sqrt{8\pi\lambda}}\$ where \$\lambda \approx 9\uparrow\uparrow (9\uparrow \uparrow (9 \uparrow \uparrow (9 \uparrow \uparrow (9\uparrow\uparrow 9)))))\$

This is g(9), approximately 10^(10^(10^(10^(10^(10^(10^8.567841344463464)))))); I'm doing g(g(g(g(g(9))))) to get the \$\lambda\$ above.

\$a \uparrow b\$ (Knuth's up-arrow notation) is a to the power of b. \$a \uparrow\uparrow b\$ is a to the power of itself, b times. So \$9\uparrow\uparrow 3 = 9^{9^9}\$ and \$9\uparrow\uparrow 9 = 9^{9^{9^{9^{9^{9^{9^{9^{9}}}}}}}}\$

from numpy.random import*
g=lambda n:n and 9**g(n-1)
print(choice([-1,1])*poisson(g(g(g(g(g(9)))))))

DON'T Try it online! Blows the recursion limit, of course.

How it works:

The Poisson distribution is a random distribution that, by itself, already generates random positive integers from 0 to infinity. This distribution takes a parameter, \$\lambda \$, that affects how likely the smaller integers are. What we have to do is set \$\lambda\$ in such a way that small numbers become very unlikely.

Then we use the wiki page again to realize the mode of the distribution is related to its parameter and then we plug the mode into the probability density function to get the program's score. After that we use Sterling's approximation to simplify the expression to something that is only slightly more digestible.

If I really can't waive the recursion limit then a similar approach with a loop gives

Python 2, score \$\approx \frac{1}{\sqrt{8\pi\lambda}}\$ where \$\lambda = 9\uparrow\uparrow (9 \uparrow\uparrow 7)\$

from numpy.random import*
print(choice([-1,1])*poisson(eval('**'.join(`9`*9**9**9**9**9**9**9**9))))

Don't try it online! Got some more bigness because @xnor taught me how join works when used directly on strings.

C (gcc), 99 bytes

f(b,t)long b(),t;{for(b()/4503599627370496&&printf("-");t-9007199254740992;t=b())putchar(t%10+48);}

b is a function returning 52 random bits in the form of an integer in the range of [0, 9007199254740992).

Try it online!

Jelly, 1 / 2M ?*,

* Any help scoring this appreciated, 0 and -1 are the two equally most likely answers at \$\frac{1}{2M}\$ - see code breakdown.

Inspiration taken from Grimmy's 05AB1E answer.

‘ɼµ⁹!!!!;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊ;”!ẋ$VƊX’µ¿~2X¤¡

No point trying this online!

Try incrementing with 4 increment tetrations (with incrementing the self-referential tetration is just doubling, also start with 2 rather than 256 so n=(2+1+1+1+1)*2*2*2*2=96, much more likely to terminate)

How?

‘ɼµ⁹!!!!;”!ẋ$VƊ;”!ẋ$VƊ ... X’µ¿~2X¤¡ - Link, no arguments
                              ¿      - while...
  µ                          µ       - ...condition: monadic chain (f(0)):
   ⁹                                 -     256
    !                                -     factorial = 256*255*...*1 = a
     !                               -     factorial = a*(a-1)*...*1 = b
      !                              -     factorial = b*(b-1)*...*1 = c
       !                             -     factorial = c*(c-1)*...*1 = A
              Ɗ                      -     last three links as a monad:
            $                        -       last two links as a monad:
         ”!                          -         literal '!' character
           ẋ                         -         repeated A times
        ;                            -       concatenate to A -> number A with A '!' after it
             V                       -       evaluate as Jelly code -> A!!..! * (A!!..!-1) * ... * 1
               ;”!ẋ$VƊ               -     call that B and repeat that process
                       ...           -     repeat 10 more times giving us M
                           X         -     choose a random integer in [1..M] inclusive
                            ’        -     decrement (i.e. while random choice != 1)
                                   ¡ - repeat...
                                  ¤  - ...number of times:
                                2    -   two
                                 X   -   random integer in [1,2] inclusive
                               ~     - ...action: bitwise NOT

05AB1E, score 0.5 / 1000!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

bug fix thanks to FryAmTheEggman

+1 ! thanks to ExpiredData

 [₄!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!LΩ#¼]¾D±)Ω

(dont't) Try it online! (will timeout).

Try it online without the factorials

[           ]            # loop:
 ₄                       #  1000
  !!...!                 #  take the factorial, 88 times
        LΩ               #  pick a random number from 1 to that many
          #              #  if the number was 1, exit the loop
           ¼             #  increment counter_variable
             ¾           # after the loop: push counter_variable
              D±         # push its bitwise negation
                )        # wrap the two in a list
                 Ω       # pick a random element of that list

Any integer would work instead of 1000!!...!, so this approach could score as low as 0.5 / BB(89), where BB is 05AB1E's Busy Beaver function.