Summary

--dport and --sport are switched.

Explanation

For the other services that I am allowing through the firewall, my machine is the server. For NTP, my machine is the client. Because the rest of my original configuration actually looked more like this:

...
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p udp --dport 5353 -j ACCEPT
...
sudo iptables -A OUTPUT -p tcp --sport 443 -j ACCEPT
sudo iptables -A OUTPUT -p tcp --sport 80 -j ACCEPT
sudo iptables -A OUTPUT -p udp --sport 5353 -j ACCEPT
...

I assumed that --dport was meant to be used with INPUT and --sport was used with OUTPUT. However, you have to think about what it means. To use NTP as a client, I need to allow INPUT packets that are coming from a source port of 123, not input packets that are coming to a destination port of 123. Likewise, I need to allow OUTPUT packets with destination port 123, not output with source 123.

So the answer to my question is to use this:

sudo iptables -P INPUT REJECT
sudo iptables -P OUTPUT REJECT
sudo iptables -P FORWARD REJECT
sudo iptables -A INPUT -p udp --sport 123 -j ACCEPT
sudo iptables -A OUTPUT -p udp --dport 123 -j ACCEPT