man 2 flock on Linux says:
[L1]
Locks created by flock() are associated with an open file description (see
open(2)). This means that duplicate file descriptors (created by, for example,
fork(2) or dup(2)) refer to the same lock, and this lock may be modified or re‐
leased using any of these file descriptors. Furthermore, the lock is released ei‐
ther by an explicit LOCK_UN operation on any of these duplicate file descriptors,
or when all such file descriptors have been closed.
[L2]
If a process uses open(2) (or similar) to obtain more than one file descriptor for
the same file, these file descriptors are treated independently by flock(). An at‐
tempt to lock the file using one of these file descriptors may be denied by a lock
that the calling process has already placed via another file descriptor.
[L3]
A process may hold only one type of lock (shared or exclusive) on a file. Subse‐
quent flock() calls on an already locked file will convert an existing lock to the
new lock mode.
As I just learned, an open file description (OFD) is the entity created when you open a file. A file descriptor is a reference to an open file description - these are the integers that you actually pass to the system calls. You can have multiple file descriptors for the same OFD.
So, I read [L1] and [L2] as saying that flocks are per-OFD. From these paragraphs I would expect that if you try to LOCK_EX the same file twice, one call will block, regardless of whether those calls are in the same process or not. I expect that if you have a LOCK_SH lock on a file, and then you try to LOCK_EX it via a different OFD, that will block, again regardless of process boundaries.
But then [L3] seems to confuse things, it would seem to say that if you have a LOCK_SH file one OFD, then open it again in the same process, then LOCK_EX the second OFD... the first lock will be converted to an exclusive lock? This doesn't make any sense to me, there are now multiple "exclusive" locks for the same file..
Is the wording just sloppy here, or am I missing something about how this system works?
The FreeBSD man page doesn't say anything about OFDs, it just says:
[B1]
Locks are on files, not file descriptors. That is, file descriptors
duplicated through dup(2) or fork(2) do not result in multiple in-
stances of a lock, but rather multiple references to a single lock. If
a process holding a lock on a file forks and the child explicitly un-
locks the file, the parent will lose its lock.
If I just trust that wording, all of my understanding of [L1] and [L2] would go out of the window and I would indeed expect LOCK_EXing a file while the same process holds a LOCK_SH on that file to just upgrade the LOCK_SH - there is no problem with multiple exclusive locks because they are just references to the same lock.
At least for Linux, I'm pretty confident that flocks are per-OFD. I think that when [L3] says "file" it really means an OFD, which I think is what @ian-abbot was saying in the comments.
Evidence:
The code looks that way (admittedly I haven't read it carefully).
I just found this bit from man fcntl (emphasis mine):
The principal difference between the two lock types is that whereas traditional record locks are associated with a process, open file description locks are associated with the open file description on which they are acquired, much like locks acquired with flock(2).
IIUC this is originally a BSD API, so unless something terrible happened the Linux API ought to be conceptually compatible with the BSD one so I think this answer almost certainly applies to BSD too.