I have followed a tutorial where it has been said that network congestion will occur when we overload our network or our maximum network capacity. In this case they referred to network capacity as our network bandwidth (maximum throughput).
So then this theoretically would mean that if a VPS provider offered unlimited bandwidth then we would never have network congestion. Is this correct?
Does network congestion occur when we exceed our network capacity?
With some qualifications ... yes.
One qualification is that a TCP connection (for example) is designed to respond to packet loss (e.g. caused by network congestion) by in effect slowing down. In effect, a TCP connection behaves in a way that tends to reduce congestion ... or at least, not make it significantly worse.
And (apparent) network congestion can occur for other reasons too. For example, it could occur due to routing problems, misconfiguration, hardware problems, other users of the network doing bad things, issues outside of the local network, and so on.
In this case they referred to network capacity as our network bandwidth (maximum throughput).
Aside: the notion of a (fixed) network bandwidth or maximum throughput is an oversimplification. (Or "advertising" ... read the fine print.) There are various things that can cause the actual throughput on a particular network path to be less than a stated (advertised) maximum. For example, if other machines or even applications on your machine are using the network, then their actions can affect the bandwidth available to your application.
So then this theoretically would mean that if a VPS provider offered unlimited bandwidth then we would never have network congestion. Is this correct?
No. It is not correct.
Firstly, VPS cannot offer unlimited bandwidth. It is not physically possible. (They may offer uncapped bandwidth1 but that typically means "as fast as the network will allow". It is not infinite bandwidth.)
Basically, every bit of hardware between your application and the application receiving the data has limits. These are inherent in the physical hardware and in the software that copies data, switches packets and controls the networking infrastructure.
Secondly, once your network packets leave the premises of the VPS, they are subject to network congestion on the internet at large, all the way to the remote servers, or remote user machines that your system is talking to. That is outside of the VPS's control.
1 - Uncapped means that the VPS should not limit what is available to your virtual machine to some predetermined agreement. But it is possible that their infrastructure may have software limits; e.g. to stop one VM on a compute node from hogging network bandwidth to the detriment of other customers. Check your VPS agreement ...