Javascript (lim(BMS),375)

See this similar answer, which uses the Bashicu Matrix System.

This is the code, where comparison is C(x)(y):

J=JSON.stringify
R='replaceAll'
N=x=>x.toString(2).split`0`.map(x=>x.length)
L=x=>x[R]('[','')[R](']','+')
E=(A,n)=>{if(!A?.[0])return "[]"
p=A.pop()
B=J(Object.assign([...p],A))
for(_=n;n--;)B=B[R](J(p),B)
return +p?J(Array(_).fill([])):B}
C=x=>y=>{M=x>y?x:y
A=[]
for(i=1;A.length<=M;i++)q='[[1]]',N(i).map(x=>q=E(eval(q),x)),A.includes(q)||A.push(q)
return L(A[x])<L(A[y])}

Analysis

Header

The header consists of four helper functions

J=JSON.stringify
R='replaceAll'
N=x=>x.toString(2).split`0`.map(x=>x.length)
L=x=>x[R]('[','')[R](']','+')

Here, N performs a bijection from a positive number into an array of nonnegative numbers by using the binary expansion. Technically, it is not a bijection since the first entry of the output array is never zero, but this works well enough for our purposes.

L is a function that strips away all of the left brackets and turn all the right brackets into pluses. This is used later for lexicographic comparison, since the + comes before , in ASCII.

Body

E=(A,n)=>{if(!A?.[0])return "[]"
p=A.pop()
B=J(Object.assign([...p],A))
for(_=n;n--;)B=B[R](J(p),B)
return +p?J(Array(_).fill([])):B}

This is basically the same thing as in a similar answer, but this time we use n specific iterations. E takes in a recursive PAS array and an expanding number n and returns the expanded string. Successor ordinal (a+1) satisfy (a+1)[n] = a

Comparison

C=x=>y=>{M=x>y?x:y
A=[]
for(i=1;A.length<=M;i++)q='[[1]]',N(i).map(x=>q=E(eval(q),x)),A.includes(q)||A.push(q)
return L(A[x])<L(A[y])}

We first set i=1 and then run through all possible natural number arrays N(i). After that, we applied those numbers as expansion numbers for the fundamental sequence of q, where q started as [[1]] = lim(BMS). After expansions, we determine whether we have seen that ordinal before or not. If we did not, we add that ordinal to the big list A.

After we generate our list of ordinals A, we compare A[x] against A[y]. PAS has a simple comparison algorithm: remove all of the left brackets and then compare lexicographically under [<,. This is what L was used for.

Sample Ordinals

A[0]  = 1     //technically we should exclude A[0], but this doesn't change the order type of the overall ordinal.
A[1]  = 0
A[2]  = ω
A[3]  = ε_0
A[4]  = 2
A[5]  = ψ(Ω_ω)
A[6]  = 4
A[7]  = ω^ω
A[8]  = (0,0,0,0)(1,1,1,1)   // this is very big ordinal
A[9]  = 3
A[10] = 8
A[11] = ω^2
A[12] = ω^(ω^(ω^ω))
A[13] = ψ(Ω_2) = BHO
A[14] = (0,0,0,0,0)(1,1,1,1,1)
A[15] = 7
A[16] = 16
A[17] = ω+1
A[18] = ω^3
A[19] = ω^(ω^(ω^2))
A[20] = ω^(ω^(ω^(ω^(ω^(ω^(ω^ω))))))
A[21] = ψ(Ω^2) = ζ_0
A[22] = ψ(Ω_4)
A[23] = (0,0,0)(1,1,1)(2,2,2) = Small Bashicu Ordinal
A[24] = (0,0,0,0,0,0)(1,1,1,1,1,1)
A[25] = 6

Charcoal, \$ω^2\$, 10 bytes

‹⟦ΣθＮ⟧⟦ΣηＮ

Try it online! Link is to verbose version of code. Orders by sum of digits, then regular ascending order, resulting in the following ordering:

1, 10, 100, ..., 2, 11, 20, 101, 200, ..., 3, 12, 21, 30, 102, 111, 120, 201, 210, 300, ..., 4, 13, 22, 31, 40, 103, 112, 121, 130, 202, 211, 220, 301, 400 ..., ...

Charcoal, \$ω^2\cdot 9\$, 14 bytes

‹⟦⌈θΣθＮ⟧⟦⌈ηΣηＮ

Try it online! Link is to verbose version of code. Orders by greatest digit, then sum of digits, then regular ascending order, resulting in the following ordering:

1, 10, 100, ..., 11, 101, ..., 111, ..., ... 2, 20, 200, ..., 12, 21, 102, 120, 201, 210, ..., 22, 112, 121, 202, 211, 220, ..., ... 9, 90, 900, ..., 91, 109, 190, 901, 910, ..., 92, 119, 191, 209, 290, 902, 911, 920, ..., ...

Charcoal, \$ω^3\$, 16 bytes

‹⟦ΣΣθΣθＮ⟧⟦ΣΣηΣηＮ

Try it online! Link is to verbose version of code. Orders by sum of digits of sum of digits, then sum of digits, then regular ascending order, resulting in the following ordering:

1, 10, ..., 19, 28, 37, 46, 55, 64, 73, 82, 91, ..., ... 2, 11, 20, ..., 29, 38, 47, 56, 65, 74, 83, 92, ..., ... 3, 12, 21, 30, ..., 39, 48, 57, 66, 75, 84, 93, ..., ... 4, 13, 22, 31, 40, ..., 49, 58, 67, 76, 85, 94, ..., ...

Charcoal, \$ω^ω\cdot 9\$, 29 bytes

Ｆ²⊞υ⟦Ｎ⟧Ｗ›⌈Ｅυ⌊κ⁹Ｆυ⊞κΣ⌊κ›⮌⊟υ⮌⊟υ

Try it online! Link is to verbose version of code. Repeatedly sums the digits of the inputs until they both reach their digital root, then compares the results in reverse order from the digital root working back to the original input. Since there are 9 digital roots and a potentially arbitrary number of summation steps I think I have the correct value for this. For small inputs the ordering is the same as the previous version; it first diverges for 199, which for instance compares less than 99 here but not in the previous ordering.

Python 2, \$\psi_0(\Omega^{\Omega^\omega})\cdot\omega\$ with respect to Buchholz's psi, 266 bytes

l=len
T=lambda i,s='(',d=1:T(i/2,s+['),','('][i%2],d+i*2%4-1)if d else(eval(s)[0],i)
L=lambda A,B:any(A==b or L(A,b)for b in B)or all(L(a,B)for a in A)and(l(A)-l(B),0)<(0,next((L(a,b)for a,b in zip(A,B)if a!=b),0))
def f(a,b):A,a=T(a);B,b=T(b);return(a,0)<(b,L(A,B))

Try it online!

At least I think that's how this ordinal is written. It's a little bigger than the Small Veblen ordinal.

T is a bijective function that maps natural numbers n to tuples (tree,remainder) of ordered trees and natural numbers. It does so by treating the reversed binary representation of the input as a sequence of (s and )s, prepended by an extra (. Once the initial ( is given a matching ), the unused bits form the remainder. For example, 37 is 100101 in binary, which gets reversed to 101001 which becomes ( ()()) 1, giving (()()) as the tree and 1 as the remainder.

L is a well ordering of ordered trees, the ordinal of which is the small Veblen ordinal. It is taken from Jervell, Herman Ruge (2005), "Finite Trees as Ordinals" (which I found via the Wikipedia article linked above). I horizontally mirrored it because it made the code shorter.

f accepts 2 natural numbers, converts them to (tree,remainder) tuples, and compares these by remainder first, then by tree.

Numbers which map to tuples in the form (tree, 0) form the small Veblen ordinal.

Tuples in the form (tree, 1) come next, then (tree, 2), (tree, 3) etc. for a total of \$\omega\$ copies of the small Veblen ordinal, which I think is written \$\phi_{\Omega^\omega}(0)\cdot\omega\$.

Note: while I defined f in terms of non-negative integers, the ordinal doesn't change if you only use strictly positive integers.

Husk, \$\omega\$, 2 bytes

¬<

Try it online!

Boring single-omega solution: well-ordering is just normal integer order (example 1 in question).

---

Husk, \$\omega\$+1, 7 bytes

¬F<σ7\0

Try it online!

Omega+1: well-ordering is integers starting at 1 in normal order, but missing 7, and finally with 7 last, so: 1,2,3,4,5,6,8,9,10,...,7.
Change the 7 in the code to select any other digit at the end.

---

Husk, \$\omega\$.2, 11 bytes

¬F<m?o_\I%2

Try it online!

Odd numbers first, then even numbers, so: 1,3,5,7,...,2,4,6,8,... .

---

Husk, \$ω^ω\$, 6 bytes

¬¤<(↔p

Try it online!

Port of Neil's omega^omega approach: gets the reversed (↔) prime factors of each ¤ argument, and compares them lexicographically (<), finally taking the logical NOT (¬) to ensure only two output values.

---

Husk, \$ω^ω\$+1, 12 bytes

¬¤>?o↔pȯ;\←←

Try it online!

Lexicographic ordering of reversed prime factors of inputs minus 1, but finally with 1 last: subtract 1 (←) from each (¤) input, then if the result is falsey (zero), convert it to a list containing infinity (ȯ;\←), otherwise make a list of its reversed (↔) prime factors, before comparing them (>) and taking the logical NOT (¬).

Polyglot, \$(\omega, 1)\$

Feel free to add languages to the list.

<

Try it in:

• Actually. Outputs 0/1.
• APL family languages: APL, J, K, BQN. Outputs 1/0.
• Jelly. Outputs 1/0.
• Keg. Outputs 0/1.
• MathGolf. Outputs 0/1.
• MATL. Outputs 1/0.
• Pyt. Outputs False/True.
• Stax. Outputs 1/0.
• Vyxal. Outputs 0/1.

Also one byte in 05AB1E, but ‹ instead of <. Outputs 0/1.

Haskell, \$(\omega, 3)\$

This solution is dead simple, really can't be beat in Haskell.

(<)

Try it online!

Haskell, \$(\omega+n,19+2\left\lfloor\log_{10}(n+1)\right\rfloor)\$

Here's a scheme to get \$\omega + n\$ for any \$n\in\mathbb{N}\$. Just replace the 9s below with one more than \$n\$. This will make the first \$n\$ numbers bigger than all other numbers but otherwise compare normally.

x#y=(x<9,x)<(y<9,y)

Try it online!

Looks like the leader board should now have an infinite number of winners. However we can beat an infinite number of these pretty easily. Most of this can just be done by fiddling with things a little bit up until we hit 26 bytes. So I will present my scores for 21, 23 and 25 without individual commentary. They should be pretty clear modifications of the 19 byte case.

21 bytes, \$\omega+98\$

x#y=(x<99,x)<(y<99,y)

23 bytes, \$\omega+387420488\$

x#y=(x<9^9,x)<(y<9^9,y)

25 bytes, \$\omega+9^{99}-1\$

(.q).(<).q
q x=(x<9^99,x)

Luckily this wraps up by 26 we can get to a much larger ordinal:

Haskell, \$(\omega\cdot n,26+\left\lfloor\log_{10}(n)\right\rfloor)\$

Thanks to the OP for pointing out that I didn't need division for this.

Another scheme much like the last, replace 9 with the desired number.

(.q).(<).q
q x=(mod x 9,x)

Try it online!

Once again we can use kolmogorov complexity to improve some of these. Here's a few that are currently records for their respective program sizes.

28 bytes, \$\omega\cdot 9^9\$

(.q).(<).q
q x=(x`mod`9^9,x)

29 bytes, \$\omega\cdot 9^{99}\$

(.q).(<).q
q x=(x`mod`9^99,x)

30 bytes, \$\omega\cdot 9^{9^9}\$

(.q).(<).q
q x=(x`mod`9^9^9,x)

31 bytes, \$\omega\cdot 9^{9^{99}}\$

(.q).(<).q
q x=(x`mod`9^9^99,x)

32 bytes, \$\omega\cdot 9^{9^{9^9}}\$

(.q).(<).q
q x=(x`mod`9^9^9^9,x)

33 bytes, \$\omega\cdot 9^{9^{9^{99}}}\$

(.q).(<).q
q x=(x`mod`9^9^9^9,x)

Haskell, \$(\omega^\omega, 63)\$

Implements the order for \$\omega^\omega\$ in the question.

n#1=[]
n#a|mod a n<1=n#div a n++[n]|m<-n+1=m#a
y=(.(2#))
f=y.y(<)

Try it online!

Haskell, \$(\phi_{\Omega^\omega}(0)\cdot\omega,208)\$

This implements the same ordinal as in cardboard_box's answer. It's slightly bigger than the small Veblen ordinal \$\phi_{\Omega^\omega}(0)\$.

This could certain be golfed more. I'm not sure that either the type or the instance are really necessary, but for now it puts Haskell back on the leaderboard.

data T=T[T]deriving Eq
instance Ord T where a<=b=a==b||a<b;T a<T b=any(T a<=)b||(all(<T b)a&&(l a,a)<(l b,b))
l=length
c x|odd x,(y,t)<-c$div x 2=(T t:)<$>c y|1>0=(div x 2,[])
g f x|(y,t)<-c x=f(T t,y)
g.g(<)

Try it online!

Haskell, \$(\phi_{\Omega^\omega}(0)^n\cdot\omega,218+2n)\$

This example code at 222 bytes, iterates the previous algorithm twice. Giving \$\phi_{\Omega^\omega}(0)^2\cdot\omega\$.

data T=T[T]deriving Eq
instance Ord T where a<=b=a==b||a<b;T a<T b=any(T a<=)b||(all(<T b)a&&(l a,a)<(l b,b))
l=length
c x|odd x,(y,t)<-c$div x 2=(T t:)<$>c y|1>0=(div x 2,[])
g f x|(y,t)<-c x=(T t,f y)
r=g$g id
f=(.r).(<).r

Try it online!

Every time we add a g$ to the front of the definition of r we add 1 to the power at the cost of two bytes.

---

Haskell Leaderboard

Since Haskell has been complete removed from the main leaderboard, for now I am maintaining a short list of Haskell winners here

Haskell, \$\omega^\omega\$,62 53 bytes

i#0=[i]
i#a=(i+1)#sum[1|'0'<-show a]++[a]
a&b=1#a<1#b

Try it online!

-9 bytes thanks to xnor

Conceptually, this transforms an integer into an infinite list, that is made by iterating sum[1|'0'<-show a], which just counts the zeroes, until zero is reached. Then it's just zeroes after that. Then those lists are then compared in reverse lexicographic order.

To actually do that in practice, the lists are constructed in reverse, and the head of the list is it's own length. The "ideal" infinite lists have an infinite trail of zeroes, which are removed in the "practical" ones. These zeroes are accounted for with the length.

For example, the number 100_000_000_001 has the "ideal" representation [100_000_000_001,10,1,0,0,0,...] and the "practical" representation [4, 1, 10, 100_000_000_001]

Pyth, \$(ω^ω, 6)\$

>_PE_P

Try it online!

Compares the reversed prime factorizations of the two inputs.

Pyth, \$(ε_0, 17)\$

L_SmylP#U_dPb>yEy

Try it online!

L                     def y(b):
           Pb           prime factors of b
   m                    map over d:
         _d               -d
        U                 range [-d, …, -1]
      P#                  filter for (negated) primes
     l                    count
    y                     recursively call y
  S                     sort
 _                      reverse
             >yEyQ    y(second input) > y(first input)

The ordinal corresponding to each positive integer is given by

$$α(p_{i_0}p_{i_1} \dotsc p_{i_{c-1}}) = \sum_k ω^{α(i_k)}$$

where \$p_i\$ is the \$i\$th prime, and the sum on the right is taken in nonincreasing order of \$α(i_k)\$ (since ordinal addition isn’t commutative). This allows us to build any ordinal with a finitely long Cantor normal form. For example:

α(1) = 0,          y(1) = []
α(2) = 1,          y(2) = [[]]
α(4) = 2,          y(4) = [[], []]
α(8) = 3,          y(8) = [[], [], []]
α(16) = 4,         y(16) = [[], [], [], []]
α(3) = ω,          y(3) = [[[]]]
α(6) = ω + 1,      y(6) = [[[]], []]
α(12) = ω + 2,     y(12) = [[[]], [], []]
α(9) = ω·2,        y(9) = [[[]], [[]]]
α(18) = ω·2 + 1,   y(18) = [[[]], [[]], []]
α(27) = ω·3,       y(27) = [[[]], [[]], [[]]]
α(7) = ω^2,        y(7) = [[[], []]]
α(14) = ω^2 + 1,   y(14) = [[[], []], []]
α(21) = ω^2 + ω,   y(21) = [[[], []], [[]]]
α(49) = ω^2·2,     y(49) = [[[], []], [[], []]]
α(19) = ω^3,       y(19) = [[[], [], []]]
α(5) = ω^ω,        y(5) = [[[[]]]]
α(10) = ω^ω + 1,   y(10) = [[[[]]], []]
α(15) = ω^ω + ω,   y(15) = [[[[]]], [[]]]
α(35) = ω^ω + ω^2, y(35) = [[[[]]], [[], []]]
α(25) = ω^ω·2,     y(25) = [[[[]]], [[[]]]]
α(13) = ω^(ω + 1), y(13) = [[[[]], []]]
α(23) = ω^(ω·2),   y(23) = [[[[]], [[]]]]
α(17) = ω^ω^2,     y(17) = [[[[], []]]]
α(11) = ω^ω^ω,     y(11) = [[[[[]]]]]
α(31) = ω^ω^ω^ω,   y(31) = [[[[[[]]]]]]

Pyth, \$(ε_0, 19)\$

L_S.ey-bkx1_jb2>yEy

Explanation for this old answer, which is now obsolete, but may be of interest for porting to languages lacking prime-related builtins.

Python 2, \$\omega^{10}\$, 48 bytes

lambda*l:cmp(*[(sorted(`n`)[::-1],n)for n in l])

Try it online!

Outputs -1 for < and +1 for >

Takes each number and sorts its digits in descending order, then compares lexicographically. This effectively compares the count of digit 9's, tiebroken by the number of count 8's, and so on down to 0's. This is all tiebroken by the value of the number itself.

This having order type \$\omega^{10}\$ follows the same argument as my Haskell \$\omega^{10}\$ answer, with sorting replacing taking the running minimum.

Haskell, \$ \omega \cdot 2\$, 22 bytes

(.q).(<).q
q=odd>>=(,)

Try it online!

Based off Grain Ghost's Haskell answers. The function q is pointfree for q x=(odd x,x). That is, we sort numbers by whether they're odd, then the number itself, which gives the evens followed by the odds. We can generalize this idea as:

Haskell, \$ \omega \cdot 4\$, 24 bytes

(.q).(<).q
q=gcd 6>>=(,)

Try it online!

Partitions numbers by the greatest common divisors with 6, which is one of [1,2,3,6]. In general, we can replace 6 with any number \$c\$, and get order type \$ \omega \cdot \sigma(c)\$, where \$ \sigma(c)\$ counts the divisors of \$c\$. For instance:

Haskell, \$ \omega \cdot 12\$, 25 bytes

(.q).(<).q
q=gcd 60>>=(,)

Try it online!

For 3 digits, 840 gives \$ \omega \cdot 32\$ for 26 bytes. The optimal values of \$c\$ to use are Highly Composite Numbers.

---

Haskell, \$\omega^{10}\$, 34 bytes

(.q).(<).q
q=scanl1 min.show>>=(,)

Try it online!

The helper scanl1 min.show takes the decimal representation of a number and computes the running minimum, resulting in a non-increasing sequence. For example, 45271 becomes 44521. We then compare these representations lexicographically, tiebroken by the number itself.

Let's show that the non-decreasing finite lists of digits \$0,\dots,9\$ are in well-ordered, with order type \$\omega^{10}\$. The lexicographic comparison first checks which sequence has more 9's in the prefix, tiebroken by whichever is followed by more 8's, and so on down to which ends with more 0's. So, if we represent the list as \$a_9\$ 9's followed by \$a_8\$ 8's, and so on to \$a_0\$ 0's, the comparison is equivalent to comparing \$a_9, a_8, \dots, a_0\$ lexicographically. Each \$a_i\$ is an arbitrary natural number, so this is exactly the order type \$\omega^{10}\$.

The scanl min operation suffices to obtain almost every non-increasing sequence of digits 0 through 9, as is seen by it leaving the numbers represented by these digits unchanged -- the only exception is sequences made of only 0's, but their omission doesn't affect the order type. Moreover, each such sequence is produced by a finite number of values, a subset of the numbers with that many digits, and it doesn't matter what the tiebreak is among these values as long as there are no ties. So, the order type produced by the main function is also \$\omega^{10}\$.

Sorting the digits in decreasing order would have also worked, but Haskell doesn't have a built-in sort. In other languages, sorting would likely be the way to go.

If we could remove the cap of 9 from the values of the digits, we'd get order type \$\omega^{\omega}\$

Python 3, \$(\omega^2,28)\$

lambda a,b:(a&-a,a)<(b&-b,b)

Try it online!

Old version

This orders by number of trailing zeros (of binary rep) first and then by the number itself.

Retina 0.8.2, \$ω^ω\$, 54 bytes

\d+
$*
+`\b(11+)(\1)+\b
$1 1$#2$*
^(.+)(,\1|\b.*,\1\B)

Try it online! Link includes test cases. Explanation:

\d+
$*

Convert to unary.

+`\b(11+)(\1)+\b
$1 1$#2$*

Factorise with the smallest prime factors last.

^(.+)(,\1|\b.*,\1\B)

Compare lexicographically.