There are [] operations which are similar to dereferencing in high-level languages. So does that mean that every variable and register name is just a pointer or are pointers a high-level languages idea and have no use in Assembly?
Pointers are a useful concept in asm, but I wouldn't say that symbol names are truly pointers. They're addresses, but they aren't pointers because there's no storage holding them (except metadata, and embedded into the machine code), and thus you can't modify them. A pointer can be incremented.
Variables are a high-level concept that doesn't truly exist in assembly. Asm has labels which you can put at any byte position in any section, including .data, .text, or whatever. Along with directives like dd, you can reserve space for a global variable and attach a symbol to it. (But a variable's value can temporarily be in a register, or for its whole lifetime if it's a local variable. The high-level concept of a variable doesn't have to map to static storage with a label.)
Types like integer vs. pointer also don't really exist in assembly; everything is just bytes that you can load into an integer, XMM, or even x87 FP register. (And you don't really have to think of that as type-punning to integer and back if you use eax to copy the bytes of a float from one memory location to another, you're just loading and storing to copy bytes around.)
But on the other hand, a pointer is a low-enough level concept still be highly relevant in assembly. We have the stack pointer (RSP) which usually holds a valid address, pointing to some memory we're using as stack space. (You can use the RSP register to hold values that aren't valid addresses, though. In that case you're not using it as a pointer. But at any time you could execute a push instruction, or mov eax, [rsp], and cause an exception from the invalid address.)
A pointer is an object that holds the address of another object. (I'm using "object" in C terms here: any byte[s] of storage that you can access, including something like an int. Not objected as in object-oriented programming.) So a pointer is basically a type of integer data, especially in assembly for a flat memory model where there aren't multiple components to it. For a segmented memory model, a seg:off far pointer is a pair of integers.
So any valid address stored anywhere in register or memory can usefully be thought of as a pointer.
But no, a symbol defined by a label is not a pointer. Conceptually, I think it's important to think of it as just a label. A pointer is itself an object (some bytes of storage), and thus can be modified. e.g. increment a pointer. But a symbol is just a way to reference some fixed position.
In C terms, a symbol is like a char symbol[], not a char *symbol = NULL; If you use bare symbol, you get the address. Like mov edi, symbol in NASM syntax. (Or mov edi, OFFSET symbol in GNU .intel_syntax or MASM. See also How to load address of function or label into register for practical considerations like using RIP-relative LEA if 32-bit absolute addresses don't work.)
You can deref any symbol in asm to access the bytes there, whether that's mov eax, [main] to load the first 4 bytes of machine code of that function, or mov eax, [global_arr + rdi*8] to index into an array, or any other x86 addressing mode. (Caveat: 32-bit absolute addresses no longer allowed in x86-64 Linux? for that last example).
But you can't do arr++; that makes no sense. There is no storage anywhere holding that address. It's embedded into the machine code of your program at each site that uses it. It's not a pointer. (Note that C arr[4] = 1 compiles to different asm depending on char *arr; vs. char arr[], but in asm you have to manually load the pointer value from wherever it's stored, and then deref it with some offset.)
If you have a label in asm whose address you want to use in C, but that isn't attached to some bytes of storage, you usually want to declare it as extern const char end_of_data_section[]; or whatever.
So for example you can do size_t data_size = data_end - data_start; and get the size in bytes as a link-time constant, if you arranged for those symbols to be at the end/start of your .data section. With a linker script or with global data_end / data_end: in your NASM source. Probably at the same address as some other symbol, for the start.