I want to understand what is usually meant by the term "OAuth Provider". There seems no mention of it in the OAuth specification.
If we consider a OAuth Provider as a component of an application, what functions does it perform (e.g. authorization, implementation of API methods, etc.)? How does it relate to the rest of the application?
Well, OAuth provider is no part of the OAuth spec, so there's no "real" answer to your question. However, it's usually the company/person who built the OAuth-protected service, OR the OAuth-protected service itself. It can also refer to the part of the API that offers OAuth endpoints (usually /authorize and /access_token).
edit: The "old" OAuth 1 community spec defined the "service provider" as
A web application that allows access via OAuth.
It was later redefined in the OAuth RFC as a server, which is:
An HTTP server (per [RFC2616]) capable of accepting OAuth-authenticated requests (Section 3).